Loading ...
Sorry, an error occurred while loading the content.

BitTorrent for Blogs

Expand Messages
  • Jeremy Bowers
    Scripting News linked to Adam Curry s review of BitTorrent; this is a review from the POV of Can it help bloggers or Userland s (or other large hosting site)
    Message 1 of 2 , Nov 30, 2002
    • 0 Attachment
      Scripting News linked to Adam Curry's review of BitTorrent; this is a
      review from the POV of "Can it help bloggers or Userland's (or other
      large hosting site) bandwidth bill?"

      Chopped a bit from the full documentation at
      http://bitconjurer.org/BitTorrent/protocol.html, BitTorrent has two
      basic pieces, the server-side "Tracker" and the client.

      Each server serving files that can be "BitTorrented" must run an
      instance of the Tracker. This tracker keeps track of who is downloading
      what, and puts the various BitTracker clients in contact with each
      other, so they can share. The Tracker and the clients coordinate amongst
      themselves to get the file to everybody, by downloading from the server
      or, preferably, directly from other clients who have the piece the
      client is looking for.

      Once the file is downloaded, the client may be closed or left open,
      depending on whether the end-user closes the program or leaves it open
      to share. Whether or not people do this will be a big variable for how
      well this works on files that lots of people access, but not necessarily
      at the same time. If the file is constantly accessed, this is less of a
      problem, as there will always be other BitTorrenters downloading it to
      share with.

      So, can it help with RSS? Recall that while the furor over bandwidth
      consumption has died down, that may yet be temporary, as all
      improvements to date are just linear and will someday also be
      overpowered. My answer to that one is a qualified "no". First, I don't
      know if BitTorrent has the equivalent of E-Tags, so it may cause a
      download on every request. That could probably be easily added, but
      perusing the code didn't show the obvious way to do it, so I don't know
      how easily. Second, no matter what, a request must start life by hitting
      the Tracker on the main server, which is going to be roughly analogous
      to an E-Tag'ed query. To improve on E-Tags and less-frequent queries
      (current state of the art), fewer requests need to be made to the RSS
      server in the first place, as in a Freenet architecture where there is
      no such thing as a "home" server. Thus, unless one has a *huge* RSS file
      that is frequently changing, I see no benefit here. (And if you have a
      *huge* RSS file that frequently changes, you should probably just turn
      down the number of items included per feed, since the RSS-reader really
      only needs *new* items.)

      However, there is frequent discussion of "multimedia blogging". I don't
      know if it will ever happen, but if it does, it will make the RSS
      bandwidth problem look positively minimal, if we're to support even
      decent quality audio, let along good quality video. BitTorrent, if
      E-Tag-style support was added to the current implementation, would
      provide a good solution for providing *lots* of *high-quality*
      multi-media blogging (i.e., large enough files that we don't have to
      have "postage stamp" video) without whacking the poor host's server too
      hard. If Userland or some other company wanted to move into that market,
      BitTorrent could cut bandwidth costs immensely, possibly significantly
      enough to provide a price advantage.

      The hooks installed into Radio for my aborted Freenet syndication
      project could be used to experiment with that on the client side, but
      BitTorrent requires some server-side installation, which I could not
      find good docs for, so I don't have any current plans to experiment with
      this. But if conditions change and multimedia blogging starts to become
      popular, I'd enjoy toying with supporting BitTorrent in a RU tool. It's
      a fun little app.

      Final note, BitTorrent is licensed with the MIT license, which is
      basically "Do whatever you want with this, as long as you keep the
      permission and copyright notice intact.", the most permissive license
      short of public domain you can get.
    • Jeremy Bowers
      ... A little more thought shows that with a slight adjustment to the protocol (since it is open) signficant RSS benefit could be obtained. Since we can kind of
      Message 2 of 2 , Nov 30, 2002
      • 0 Attachment
        On Sat, 2002-11-30 at 13:07, Jeremy Bowers wrote:
        > So, can it help with RSS? Recall that while the furor over bandwidth
        > consumption has died down, that may yet be temporary, as all
        > improvements to date are just linear and will someday also be
        > overpowered. My answer to that one is a qualified "no".

        A little more thought shows that with a slight adjustment to the
        protocol (since it is open) signficant RSS benefit could be obtained.
        Since we can kind of count on the BitTorrent app being open long-term,
        we could make the following changes:

        * The Tracker elects some "trusted nodes", based on connectivity, some
        randomness, and the total activity of the file.
        * When a client downloads an RSS file for the first time, the Tracker
        sends down some part of the trusted node list (two or three random
        nodes).
        * The Tracker notifies the trusted nodes when the file changes (which is
        also an opportunity to update the trusted node list, if they don't
        respond).
        * The clients ask the trusted nodes whether the file has changed,
        instead of asking the server directly. Since we're talking about leaving
        things open, we can also try to cache what other nodes are available and
        only ask the Tracker for other available nodes to share from if the
        client runs out of connected nodes.

        With this slight change, one request to the main web server (which is
        the bandwidth that we are really worried about) can result in multiple
        RSS files being transferred, as long as the trusted nodes are
        well-behaved. Depending on how well-behaved, the trusted nodes may be
        able to act as a server proxy for many hours or even days. This *is* an
        actual qualitative improvement to RSS serving, as the burden on the main
        server goes to one hit per *consumer* plus a log(consumer) maintainence
        fee, based on how well-behaved the trusted nodes are and how much we fan
        the duties out. (In constrast to the current one hit per consumer per
        update.... essentially going from n^2 to n log n.)
      Your message has been successfully submitted and would be delivered to recipients shortly.