Loading ...
Sorry, an error occurred while loading the content.

Re: p2p working group/standards

Expand Messages
  • allenjs@hotmail.com
    ... I just had a chance to read through this, and it looks like an interesting addition to the pool of options. Some comments: You mention the fact that
    Message 1 of 55 , Feb 15, 2001
    • 0 Attachment
      > A bunch of us at U.C.Berkeley and ACIRI (AT&T Center for Internet
      > Research) have recently been working on designing a highly
      > scalable, distributed indexing system.
      >
      > I've attached a copy of a paper (currently under submission to
      > the ACM Sigcomm conference) describing our algorithm.
      > We would love to get feedback on our work from the people
      > on this list.
      >

      I just had a chance to read through this, and it looks like an
      interesting addition to the pool of options. Some comments:

      You mention the fact that Napster has a centralized directory as
      being a problem for scalability. Actually, napster is scalable
      *because* it has a centralized metadata directory. Even in the
      scheme documented in your paper, performance improves drastically as
      the machines are moved closer together. Certainly you could say that
      your technique would scale when confronted with enough data to make a
      large datacenter keel over, but until the point that a centralized
      system becomes physically impossible, centralized metadata scales
      better. (Incidentally, since Google has a copy of every page it
      scans cached centrally, and moore's law is faster than us, maybe
      centralizing will continue to be practical). And I cannot argue with
      your assertion that centralized directory is vulnerable. That is
      true.

      I noticed that you referred to web caching schemes, which seems a
      very appropriate comparison. I believe that the most likely scheme
      to evolve will be a situation where semantic information (metadata,
      song titles, etc.) flow to central directories, but chunks of those
      directories get cached closer to users based on demand. I outlined
      this idea in the document at
      http://www.netcrucible.com/semantic.html. And I do believe that the
      demand-based dynamic caching of directory information will ultimately
      be handled in a way similar to what you describe. A very similar
      idea (but optimized for latency in the other direction) was proposed
      by IBM at WWW9 - http://www9.org/w9cdrom/301/301.html - this paper
      also does a really good job of outlining the other various
      distributed hash-based lookups available. In any case, I am in
      agreement with you that this line of research being pursued by akamai
      and inktomi will be beneficial to the ability to distribute directory
      in the future.

      Finally, the hashtable approach in a query is only going to allow
      lookups that are an exact match (which is why it's appropriate for
      web caches, where the URL is consistent). For example, doing a
      substring search (as napster does with filenames) would be impossible
      or at least impractical with this method. There are certainly
      techniques to attack this problem as well, though..
    • Ben Houston
      ... Probably going to SHA-1 isn t too big of a problem. I ll bring it up with those that I know. Interestingly, there are ways to add file hashes within the
      Message 55 of 55 , Feb 20, 2001
      • 0 Attachment
        Justin Chapweske said:
        > ... switch to SHA-1.
        >
        > The biggest group that I havn't yet talked to about this is the Gnutella
        > guys, but I'm sure they'd be into it as well. Any Gnutellians on the
        > list?

        Probably going to SHA-1 isn't too big of a problem. I'll bring it up with
        those that I know. Interestingly, there are ways to add file hashes within
        the existing protocol specifications - it should even be backwards
        compatible.

        Cheers,
        -ben houston
        http://www.exocortex.org/~ben
      Your message has been successfully submitted and would be delivered to recipients shortly.