Loading ...
Sorry, an error occurred while loading the content.

RE: Http overhead

Expand Messages
  • Joris Dobbelsteen
    Many apache servers send data chunked. Takes a couple bytes (average of 4) for every block transfered. Maybe a total overhead of an additional 20-40 bytes per
    Message 1 of 11 , Dec 1, 2000
    • 0 Attachment
      Many apache servers send data chunked. Takes a couple bytes (average of
      4) for every block transfered. Maybe a total overhead of an additional
      20-40 bytes per transfer (maybe less). This is just a guess...


      - Joris


      > -----Original Message-----
      > From: francis@...
      > [mailto:francis@...]On Behalf Of John Stracke
      > Sent: Thursday 30 November, 2000 16:45
      > To: http-wg@...
      > Subject: Re: Http overhead
      >
      >
      > dillon@... wrote:
      >
      > > The latest standard (HTTP 1.1) has provisions for
      > compression and "chunked"
      > > transfers which change this, but I haven't seen these used
      > in any real-world
      > > situations
      > > yet.
      >
      > Apache will recognize a file with a ".gz" extension as
      > gzipped, and send the
      > Content-Encoding: x-gzip line. Netscape will recognize
      > Content-Encoding: x-gzip,
      > and uncompress the file. Unfortunately, at least in my
      > installation (Apache
      > 1.3.14, Red Hat 7), Apache doesn't look at the extension
      > before the ".gz" to get
      > the content-type; "foo.txt.gz" gets marked as Content-Type:
      > application/x-gzip.
      >
      > --
      > /=================================================================\
      > |John Stracke | http://www.ecal.com |My opinions are my own. |
      > |Chief Scientist |================================================|
      > |eCal Corp. |But how do we know destroying the Van Allen belt|
      > |francis@...|will kill all life on Earth if we don't try it? |
      > \=================================================================/
      >
      >
      >
      >
    • John Stracke
      ... Yes, but it s actually better than that: AFAIK, Apache uses chunked transfer-encoding only for dynamic resources, where it can t predict the
      Message 2 of 11 , Dec 1, 2000
      • 0 Attachment
        Joris Dobbelsteen wrote:

        > Many apache servers send data chunked. Takes a couple bytes (average of
        > 4) for every block transfered. Maybe a total overhead of an additional
        > 20-40 bytes per transfer (maybe less). This is just a guess...

        Yes, but it's actually better than that: AFAIK, Apache uses chunked
        transfer-encoding only for dynamic resources, where it can't predict the
        content-length. The alternative would be (a) buffer the output before
        sending it down, or (b) defeat persistent connections. Either (a) or (b)
        would increase; (b) would actually cost extra bandwidth, and (a) would
        cause bandwidth consumption to come in spikes. So, most likely, the cost
        of chunking is lower than the cost of not chunking; it's certainly lower
        than the nominal overhead of the encoding.

        (Sorry to those to whom this is obvious--probably including Joris--but I
        didn't want to leave anybody thinking they could save bandwidth by turning
        off chunking. :-)

        --
        /=================================================================\
        |John Stracke | http://www.ecal.com |My opinions are my own. |
        |Chief Scientist |================================================|
        |eCal Corp. |In the country of the blind, the one-eyed man is|
        |francis@...|in therapy. |
        \=================================================================/
      • Joris Dobbelsteen
        You are right, I did know this. Indeed that s why I favor chunked encoding for dynamic resources. MSN s (search) server (MS-IIS) transfers the data by just
        Message 3 of 11 , Dec 1, 2000
        • 0 Attachment
          You are right, I did know this.

          Indeed that's why I favor chunked encoding for dynamic resources. MSN's
          (search) server (MS-IIS) transfers the data by just closing the
          connection at the end (not sending any indication about the size of the
          document) what also decreases the transfer 'reliability' - or better,
          you don't know wether you have all or not... HTML or JPEG/GIF you can
          guess it, but with many other sources you don't (like TXT)....

          Also MSN's server has thus more overhead for the server and waisted time
          between the requests for the client...

          - Joris

          > -----Original Message-----
          > From: francis@...
          > [mailto:francis@...]On Behalf Of John Stracke
          > Sent: Friday 01 December, 2000 16:04
          > To: WWW WG (E-mail)
          > Subject: Re: Http overhead
          >
          >
          > Joris Dobbelsteen wrote:
          >
          > > Many apache servers send data chunked. Takes a couple bytes
          > (average of
          > > 4) for every block transfered. Maybe a total overhead of an
          > additional
          > > 20-40 bytes per transfer (maybe less). This is just a guess...
          >
          > Yes, but it's actually better than that: AFAIK, Apache uses chunked
          > transfer-encoding only for dynamic resources, where it can't
          > predict the
          > content-length. The alternative would be (a) buffer the output before
          > sending it down, or (b) defeat persistent connections.
          > Either (a) or (b)
          > would increase; (b) would actually cost extra bandwidth, and (a) would
          > cause bandwidth consumption to come in spikes. So, most
          > likely, the cost
          > of chunking is lower than the cost of not chunking; it's
          > certainly lower
          > than the nominal overhead of the encoding.
          >
          > (Sorry to those to whom this is obvious--probably including
          > Joris--but I
          > didn't want to leave anybody thinking they could save
          > bandwidth by turning
          > off chunking. :-)
          >
          > --
          > /=================================================================\
          > |John Stracke | http://www.ecal.com |My opinions are my own. |
          > |Chief Scientist |================================================|
          > |eCal Corp. |In the country of the blind, the one-eyed man is|
          > |francis@...|in therapy. |
          > \=================================================================/
          >
          >
          >
          >
        • Joris Dobbelsteen
          It did was a while ago, when I did some experimenting (personal purpose) with HTTP. Your server (MSN Search) didn t return a content-length field or send the
          Message 4 of 11 , Dec 2, 2000
          • 0 Attachment
            It did was a while ago, when I did some experimenting (personal purpose)
            with HTTP. Your server (MSN Search) didn't return a content-length field
            or send the data chunked (this is a good while ago)... Also another
            server from CNET did the same thing. At least this was about a year ago,
            or maybe even a longer time a ago, this wasn't recently....

            I don't know how your search servers work, but maybe the server just
            forwarded a response from another server. The URL I gave to the server
            (using telnet) I had from the search area in IE 4. So maybe actual data
            came from someone like Infoseek or Aslavista....
            The server however did turn, at that time, "Server: Microsoft-IIS[...]".

            At that time I was developing a simple proxy server (for myself), that
            ran into trouble with sites not giving a content-length or using chunked
            tranfer-encoding. The first trouble I then developed support was chunked
            tranfer encoding, what did turn out to work correct. But at this time I
            still couldn't download from MSN search of CNET. Possiblity that it was
            this error.

            Also the proxy didn't support caching (yet) and did turn out to be quite
            unreliable, especially with subsequent requests on a connection.


            But don't forget I is a long while ago, what I actually should have put
            in the previous mail (my fault). Software and systems evolve, and you
            probably don't have the same stuff you had a couple years ago...


            - Joris


            > -----Original Message-----
            > From: Chris Wendt [mailto:christw@...]
            > Sent: Saturday 02 December, 2000 5:12
            > To: 'joris.dobbelsteen@...'
            > Subject: RE: Http overhead
            >
            >
            > Joris,
            >
            > We in MSN Search do populate the content length field in the
            > header, just
            > verified. Could you point out what you consider wrong in what you are
            > seeing?
            >
            > In general, with IIS, if the client is HTTP 1.0 (old), and
            > the server ASP
            > dumps the buffer before the response ends, then there may be a chunked
            > response. I am not aware of our service ever invoking that
            > behavior. This
            > should be the same behavior for any web server.
            >
            > Chris..
            >
            > -----Original Message-----
            > From: Joris Dobbelsteen [mailto:joris.dobbelsteen@...]
            > Sent: Friday, December 01, 2000 1:24 PM
            > To: WWW WG (E-mail)
            > Subject: RE: Http overhead
            >
            > You are right, I did know this.
            > Indeed that's why I favor chunked encoding for dynamic
            > resources. MSN's
            > (search) server (MS-IIS) transfers the data by just closing the
            > connection at the end (not sending any indication about the
            > size of the
            > document) what also decreases the transfer 'reliability' - or better,
            > you don't know wether you have all or not... HTML or JPEG/GIF you can
            > guess it, but with many other sources you don't (like TXT)....
            > Also MSN's server has thus more overhead for the server and
            > waisted time
            > between the requests for the client...
            > - Joris
            > > -----Original Message-----
            > > From: francis@...
            > > [mailto:francis@...]On Behalf Of John Stracke
            > > Sent: Friday 01 December, 2000 16:04
            > > To: WWW WG (E-mail)
            > > Subject: Re: Http overhead
            > >
            > >
            > > Joris Dobbelsteen wrote:
            > >
            > > > Many apache servers send data chunked. Takes a couple bytes
            > > (average of
            > > > 4) for every block transfered. Maybe a total overhead of an
            > > additional
            > > > 20-40 bytes per transfer (maybe less). This is just a guess...
            > >
            > > Yes, but it's actually better than that: AFAIK, Apache uses chunked
            > > transfer-encoding only for dynamic resources, where it can't
            > > predict the
            > > content-length. The alternative would be (a) buffer the
            > output before
            > > sending it down, or (b) defeat persistent connections.
            > > Either (a) or (b)
            > > would increase; (b) would actually cost extra bandwidth,
            > and (a) would
            > > cause bandwidth consumption to come in spikes. So, most
            > > likely, the cost
            > > of chunking is lower than the cost of not chunking; it's
            > > certainly lower
            > > than the nominal overhead of the encoding.
            > >
            > > (Sorry to those to whom this is obvious--probably including
            > > Joris--but I
            > > didn't want to leave anybody thinking they could save
            > > bandwidth by turning
            > > off chunking. :-)
            > >
            > > --
            > > /=================================================================\
            > > |John Stracke | http://www.ecal.com |My opinions are my own. |
            > > |Chief Scientist |================================================|
            > > |eCal Corp. |In the country of the blind, the one-eyed man is|
            > > |francis@...|in therapy. |
            > > \=================================================================/
            > >
            > >
            > >
            > >
            >
          Your message has been successfully submitted and would be delivered to recipients shortly.