Loading ...
Sorry, an error occurred while loading the content.

mod_perl2, HEAD request and Content-Length

Expand Messages
  • Boris Zentner
    Hi, I have a handler, that serve dynamic pages or static ones. If the handler gets a HEAD request, it answers with ... $apr- headers_out- { Content-Length } =
    Message 1 of 25 , Jul 30, 2004
    • 0 Attachment
      Hi,

      I have a handler, that serve dynamic pages or static ones. If the handler gets
      a HEAD request, it answers with

      ...
      $apr->headers_out->{'Content-Length'} = 12345;
      $apr->content_type($media_type) unless $apr->main;
      if ( $apr->header_only ) {
      return DONE;
      }
      ...

      This works, but the Content-Length header is removed by someone. I do not use
      any filters and even when, there is no body of the message.

      Who strips my Content-Length header?

      How to find who modified my headers?

      Is this a bug or a desired behavior?

      I know there are filters and perhaps other handlers and for some of them it
      makes sense to modify or strip my header, but is that true in my case?

      Currently I use rflush to force out the headers. What is the recommended
      practice?

      ...
      $apr->headers_out->{'Content-Length'} = 12345;
      $apr->content_type($media_type) unless $apr->main;
      if ( $apr->header_only ) {
      $apr->rflush;
      return DONE;
      }
      ...

      --
      Boris

      --
      Report problems: http://perl.apache.org/bugs/
      Mail list info: http://perl.apache.org/maillist/modperl.html
      List etiquette: http://perl.apache.org/maillist/email-etiquette.html
    • Geoffrey Young
      ... it could be anyone, from some C filter (like mod_include) to apache core. ... use the force, read the source :) ... it is desired - you no longer need to
      Message 2 of 25 , Jul 30, 2004
      • 0 Attachment
        Boris Zentner wrote:
        > Hi,
        >
        > I have a handler, that serve dynamic pages or static ones. If the handler gets
        > a HEAD request, it answers with
        >
        > ...
        > $apr->headers_out->{'Content-Length'} = 12345;
        > $apr->content_type($media_type) unless $apr->main;
        > if ( $apr->header_only ) {
        > return DONE;
        > }
        > ...
        >
        > This works, but the Content-Length header is removed by someone. I do not use
        > any filters and even when, there is no body of the message.
        >
        > Who strips my Content-Length header?

        it could be anyone, from some C filter (like mod_include) to apache core.

        >
        > How to find who modified my headers?

        use the force, read the source :)

        >
        > Is this a bug or a desired behavior?

        it is desired - you no longer need to set the Content-Length header for
        requests in Apache 2.0.

        basically, any C-L header you set in the content-generation phase of the
        request has the potential to be incorrect, since any filter can come along
        and alter the content. so, what apache does is it runs the new
        content-length filter, which will decide whether a C-L header is required
        and take appropriate steps. for instance, apache will abandon the C-L
        header if it decides it can use a chunked transfer encoding instead.

        HTH

        --Geoff

        --
        Report problems: http://perl.apache.org/bugs/
        Mail list info: http://perl.apache.org/maillist/modperl.html
        List etiquette: http://perl.apache.org/maillist/email-etiquette.html
      • Stas Bekman
        ... [...] ... It s somewhat documented: http://perl.apache.org/docs/2.0/user/handlers/http.html#Handling_HEAD_Requests ... Geoff, feel free to improve that
        Message 3 of 25 , Jul 31, 2004
        • 0 Attachment
          Geoffrey Young wrote:
          >
          > Boris Zentner wrote:
          >
          >>Hi,
          >>
          >>I have a handler, that serve dynamic pages or static ones. If the handler gets
          >>a HEAD request, it answers with
          [...]
          > it is desired - you no longer need to set the Content-Length header for
          > requests in Apache 2.0.

          It's somewhat documented:
          http://perl.apache.org/docs/2.0/user/handlers/http.html#Handling_HEAD_Requests

          > basically, any C-L header you set in the content-generation phase of the
          > request has the potential to be incorrect, since any filter can come along
          > and alter the content. so, what apache does is it runs the new
          > content-length filter, which will decide whether a C-L header is required
          > and take appropriate steps. for instance, apache will abandon the C-L
          > header if it decides it can use a chunked transfer encoding instead.

          Geoff, feel free to improve that section to be more complete. Thanks.

          --
          __________________________________________________________________
          Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
          http://stason.org/ mod_perl Guide ---> http://perl.apache.org
          mailto:stas@... http://use.perl.org http://apacheweek.com
          http://modperlbook.org http://apache.org http://ticketmaster.com

          --
          Report problems: http://perl.apache.org/bugs/
          Mail list info: http://perl.apache.org/maillist/modperl.html
          List etiquette: http://perl.apache.org/maillist/email-etiquette.html
        • Boris Zentner
          Hi, ... I just read it, thanks for the pointer. I think the text should make clear, that even if it is not needed to return early with return OK if
          Message 4 of 25 , Aug 1, 2004
          • 0 Attachment
            Hi,

            Am Samstag 31 Juli 2004 10:20 schrieb Stas Bekman:
            > Geoffrey Young wrote:
            > > Boris Zentner wrote:
            > >>Hi,
            > >>
            > >>I have a handler, that serve dynamic pages or static ones. If the handler
            > >> gets a HEAD request, it answers with
            >
            > [...]
            >
            > > it is desired - you no longer need to set the Content-Length header for
            > > requests in Apache 2.0.
            >
            > It's somewhat documented:
            > http://perl.apache.org/docs/2.0/user/handlers/http.html#Handling_HEAD_Reque
            >sts
            >

            I just read it, thanks for the pointer. I think the text should make clear,
            that even if it is not needed to return early with 'return OK if
            $r->header_only;' with apache2, it is recommended, since it is ( or can be )
            a big waste of time to construct the body, that is removed in the next pass.

            From reading the comments in modules/http/http_protocol.c
            ( ap_http_header_filter ), I get the impression, that apache2 recommend
            generating and throwing away the content. Here is the comment in question:

            /* This is a hack, but I can't find anyway around it. The idea is that
            * we don't want to send out 0 Content-Lengths if it is a head request.
            * This happens when modules try to outsmart the server, and return
            * if they see a HEAD request. Apache 1.3 handlers were supposed to
            * just return in that situation, and the core handled the HEAD. In
            * 2.0, if a handler returns, then the core sends an EOS bucket down
            * the filter stack, and the content-length filter computes a C-L of
            * zero and that gets put in the headers, and we end up sending a
            * zero C-L to the client. We can't just remove the C-L filter,
            * because well behaved 2.0 handlers will send their data down the stack,
            * and we will compute a real C-L for the head request. RBB
            */

            also The comment implies, that I get a Content-Length header for a HEAD
            request, if I waste my time and generate the body of the message. I tried it
            and got _no_ Content-Length header!

            $apr->content_type($media_type) unless $apr->main;
            if ( $apr->header_only ) {
            $apr->print( ' ' x $size ); # just to see if I got a content-length header
            return DONE;
            }

            The only way to get the Content-Length header for a HEAD request is

            $apr->headers_out->{'Content-Length'} = $size ;
            $apr->content_type($media_type) unless $apr->main;
            if ( $apr->header_only ) {
            $apr->rflush;
            return DONE;
            }

            But I fail to see why this works, I thought the data walks the same filter
            chain only in more buckets.

            I used

            Server: Apache/2.0.49 (Unix) mod_perl/1.99_14 Perl/v5.8.3 mod_ssl/2.0.49
            OpenSSL/0.9.7d DAV/2 SVN/1.0.2

            > > basically, any C-L header you set in the content-generation phase of the
            > > request has the potential to be incorrect, since any filter can come
            > > along and alter the content. so, what apache does is it runs the new
            > > content-length filter, which will decide whether a C-L header is required
            > > and take appropriate steps. for instance, apache will abandon the C-L
            > > header if it decides it can use a chunked transfer encoding instead.
            >
            > Geoff, feel free to improve that section to be more complete. Thanks.
            >

            --
            Boris

            --
            Report problems: http://perl.apache.org/bugs/
            Mail list info: http://perl.apache.org/maillist/modperl.html
            List etiquette: http://perl.apache.org/maillist/email-etiquette.html
          • Stas Bekman
            ... That s what I understood as well. Does anybody have any statistics on the percentage of HEAD requests vs. GET? The purpose of HEAD request is not to save
            Message 5 of 25 , Aug 1, 2004
            • 0 Attachment
              Boris Zentner wrote:
              > Hi,
              >
              > Am Samstag 31 Juli 2004 10:20 schrieb Stas Bekman:
              >
              >>Geoffrey Young wrote:
              >>
              >>>Boris Zentner wrote:
              >>>
              >>>>Hi,
              >>>>
              >>>>I have a handler, that serve dynamic pages or static ones. If the handler
              >>>>gets a HEAD request, it answers with
              >>
              >>[...]
              >>
              >>
              >>>it is desired - you no longer need to set the Content-Length header for
              >>>requests in Apache 2.0.
              >>
              >>It's somewhat documented:
              >>http://perl.apache.org/docs/2.0/user/handlers/http.html#Handling_HEAD_Reque
              >>sts
              >>
              >
              >
              > I just read it, thanks for the pointer. I think the text should make clear,
              > that even if it is not needed to return early with 'return OK if
              > $r->header_only;' with apache2, it is recommended, since it is ( or can be )
              > a big waste of time to construct the body, that is removed in the next pass.
              >
              > From reading the comments in modules/http/http_protocol.c
              > ( ap_http_header_filter ), I get the impression, that apache2 recommend
              > generating and throwing away the content. Here is the comment in question:
              >
              > /* This is a hack, but I can't find anyway around it. The idea is that
              > * we don't want to send out 0 Content-Lengths if it is a head request.
              > * This happens when modules try to outsmart the server, and return
              > * if they see a HEAD request. Apache 1.3 handlers were supposed to
              > * just return in that situation, and the core handled the HEAD. In
              > * 2.0, if a handler returns, then the core sends an EOS bucket down
              > * the filter stack, and the content-length filter computes a C-L of
              > * zero and that gets put in the headers, and we end up sending a
              > * zero C-L to the client. We can't just remove the C-L filter,
              > * because well behaved 2.0 handlers will send their data down the stack,
              > * and we will compute a real C-L for the head request. RBB
              > */

              That's what I understood as well.

              Does anybody have any statistics on the percentage of HEAD requests vs.
              GET? The purpose of HEAD request is not to save your CPU cycles, but to
              avoid data transfer which is precisely what Apache 2.0 does, without
              making you do the extra work to deal with HEAD requests.

              > also The comment implies, that I get a Content-Length header for a HEAD
              > request, if I waste my time and generate the body of the message. I tried it
              > and got _no_ Content-Length header!
              >
              > $apr->content_type($media_type) unless $apr->main;
              > if ( $apr->header_only ) {
              > $apr->print( ' ' x $size ); # just to see if I got a content-length header
              > return DONE;
              > }
              >
              > The only way to get the Content-Length header for a HEAD request is
              >
              > $apr->headers_out->{'Content-Length'} = $size ;
              > $apr->content_type($media_type) unless $apr->main;
              > if ( $apr->header_only ) {
              > $apr->rflush;
              > return DONE;
              > }
              >
              > But I fail to see why this works, I thought the data walks the same filter
              > chain only in more buckets.

              In dynamic scripts you never get the C-L header unless you set it
              explicitly and some filter doesn't remove it. It's not possible for
              Apache to add one, since headers are sent *before* it knows how much
              data your handler is going to send. So HEAD or GET, you won't get it.

              So HEAD on dynamic scripts that doesn't set their own C-L is a wasted
              operation on behalf of the client. Unless you use some E-Tag (but I
              haven't checked the RFC).

              BTW, what's wrong with $r->set_content_length()?
              http://perl.apache.org/docs/2.0/api/Apache/Response.html#C_set_content_length_
              Neither you need to rflush. Neither you should return DONE, but OK.

              --
              __________________________________________________________________
              Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
              http://stason.org/ mod_perl Guide ---> http://perl.apache.org
              mailto:stas@... http://use.perl.org http://apacheweek.com
              http://modperlbook.org http://apache.org http://ticketmaster.com

              --
              Report problems: http://perl.apache.org/bugs/
              Mail list info: http://perl.apache.org/maillist/modperl.html
              List etiquette: http://perl.apache.org/maillist/email-etiquette.html
            • Boris Zentner
              Hi, ... But my point is, for a HEAD request, there is no data so apache should not touch my content-length header. I really dislike to generate the full data
              Message 6 of 25 , Aug 1, 2004
              • 0 Attachment
                Hi,

                Am Sonntag 01 August 2004 18:13 schrieb Stas Bekman:
                > Does anybody have any statistics on the percentage of HEAD requests vs.
                > GET? The purpose of HEAD request is not to save your CPU cycles, but to
                > avoid data transfer which is precisely what Apache 2.0 does, without
                > making you do the extra work to deal with HEAD requests.
                >
                > > also The comment implies, that I get a Content-Length header for a HEAD
                > > request, if I waste my time and generate the body of the message. I tried
                > > it and got _no_ Content-Length header!
                > >
                > >  $apr->content_type($media_type) unless $apr->main;
                > >   if ( $apr->header_only ) {
                > >     $apr->print( ' ' x $size ); # just to see if I got a content-length
                > > header return DONE;
                > >   }
                > >
                > > The only way to get the Content-Length header for a HEAD request is
                > >
                > >  $apr->headers_out->{'Content-Length'} = $size ;
                > >  $apr->content_type($media_type) unless $apr->main;
                > >   if ( $apr->header_only ) {
                > >     $apr->rflush;
                > >     return DONE;
                > >   }
                > >
                > > But I fail to see why this works, I thought the data walks the same
                > > filter chain only in more buckets.
                >
                > In dynamic scripts you never get the C-L header unless you set it
                > explicitly and some filter doesn't remove it. It's not possible for
                > Apache to add one, since headers are sent *before* it knows how much
                > data your handler is going to send. So HEAD or GET, you won't get it.

                But my point is, for a HEAD request, there is no data so apache should not
                touch my content-length header. I really dislike to generate the full data
                for the request and apache throws it away ( and even the I get no
                Content-Length header ).

                >
                > So HEAD on dynamic scripts that doesn't set their own C-L is a wasted
                > operation on behalf of the client. Unless you use some E-Tag (but I
                > haven't checked the RFC).
                >
                > BTW, what's wrong with $r->set_content_length()?

                Nothing, I searched for the function, but I did not find it! I know I should
                use mp2doc, but as I searched for it, I did not think on the tool. I tryed
                first with perldoc Apache::RequestRec and then I searched

                http://perl.apache.org/docs/2.0/api/Apache/RequestRec.html

                For some reason I thought if there is no function content_length then I have
                no function like this.

                > http://perl.apache.org/docs/2.0/api/Apache/Response.html#C_set_content_leng
                >th_ Neither you need to rflush. Neither you should return DONE, but OK.

                I tried it now and even with set_content_length and return OK I got no
                content-length header.

                PS: Much thanks for the pointer to Apache::Response, I found more functions
                that I missed already!

                --
                Boris

                --
                Report problems: http://perl.apache.org/bugs/
                Mail list info: http://perl.apache.org/maillist/modperl.html
                List etiquette: http://perl.apache.org/maillist/email-etiquette.html
              • Stas Bekman
                Boris Zentner wrote: [...] ... (sure, it was unrelated to the problem you were mentioned) let me try to write a test and I ll be back to you. Me thinking that
                Message 7 of 25 , Aug 1, 2004
                • 0 Attachment
                  Boris Zentner wrote:
                  [...]
                  > Nothing, I searched for the function, but I did not find it! I know I should
                  > use mp2doc, but as I searched for it, I did not think on the tool. I tryed
                  > first with perldoc Apache::RequestRec and then I searched
                  >
                  > http://perl.apache.org/docs/2.0/api/Apache/RequestRec.html
                  >
                  > For some reason I thought if there is no function content_length then I have
                  > no function like this.
                  >
                  >
                  >>http://perl.apache.org/docs/2.0/api/Apache/Response.html#C_set_content_leng
                  >>th_ Neither you need to rflush. Neither you should return DONE, but OK.
                  >
                  >
                  > I tried it now and even with set_content_length and return OK I got no
                  > content-length header.

                  (sure, it was unrelated to the problem you were mentioned)

                  let me try to write a test and I'll be back to you.

                  Me thinking that we should drop the /^set_/ prefix in
                  set_content_length, to be consistent with the rest of the API.

                  > PS: Much thanks for the pointer to Apache::Response, I found more functions
                  > that I missed already!

                  All Apache::RequestRec classes now xref themselves from their
                  Description sections, e.g.:
                  http://perl.apache.org/docs/2.0/api/Apache/RequestRec.html#Description

                  --
                  __________________________________________________________________
                  Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                  http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                  mailto:stas@... http://use.perl.org http://apacheweek.com
                  http://modperlbook.org http://apache.org http://ticketmaster.com

                  --
                  Report problems: http://perl.apache.org/bugs/
                  Mail list info: http://perl.apache.org/maillist/modperl.html
                  List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                • Stas Bekman
                  ... I get indentical behavior with either of the two methods: my $body = This is a response string ; #$r- headers_out- { Content-Length } = length $body;
                  Message 8 of 25 , Aug 1, 2004
                  • 0 Attachment
                    Stas Bekman wrote:

                    > let me try to write a test and I'll be back to you.

                    I get indentical behavior with either of the two methods:

                    my $body = "This is a response string";
                    #$r->headers_out->{'Content-Length'} = length $body;
                    $r->set_content_length(length $body);

                    $r->print('a');

                    if the one-char-print line is commented out, I *never* get the C-L
                    header for HEAD requests. Otherwise I always get the correct C-L header.

                    Normally, Apache is not able to figure out how much data the content
                    handler is going to send, since it needs to send headers early. However
                    in the case where your handler returns w/o sending anything at all, the
                    headers_out filter sees EOS and no data, so it probably thinks that C-L
                    is wrong and drops it. My test with printing just one char makes it
                    working. It's safe to send anything, since Apache will discard it for
                    HEAD requests anyway. So here you have a workaround (but I can't promise
                    that it will always continue to work).

                    Please ask at the Apache list, whether there is a cleaner way to
                    make Apache preserve the C-L header, w/o sending any data, and please
                    let us know the verdict.

                    p.s. I'm not sure how you've got a HEAD request with C-L header working,
                    when using $r->headers_out->{'Content-Length'}; it doesn't work for me.
                    (though I use Apache 2.0.50-dev so I won't be surprised if something has
                    changed).

                    --
                    __________________________________________________________________
                    Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                    http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                    mailto:stas@... http://use.perl.org http://apacheweek.com
                    http://modperlbook.org http://apache.org http://ticketmaster.com

                    --
                    Report problems: http://perl.apache.org/bugs/
                    Mail list info: http://perl.apache.org/maillist/modperl.html
                    List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                  • Stas Bekman
                    ... I ve just committed this test to the modperl-2.0 cvs rep: t/apache/head_request.t t/response/TestApache/head_request.pm please take a look --
                    Message 9 of 25 , Aug 1, 2004
                    • 0 Attachment
                      Stas Bekman wrote:
                      > Stas Bekman wrote:
                      >
                      >> let me try to write a test and I'll be back to you.

                      I've just committed this test to the modperl-2.0 cvs rep:
                      t/apache/head_request.t t/response/TestApache/head_request.pm
                      please take a look

                      --
                      __________________________________________________________________
                      Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                      http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                      mailto:stas@... http://use.perl.org http://apacheweek.com
                      http://modperlbook.org http://apache.org http://ticketmaster.com

                      --
                      Report problems: http://perl.apache.org/bugs/
                      Mail list info: http://perl.apache.org/maillist/modperl.html
                      List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                    • Geoffrey Young
                      ... for the record, this is fundamentally wrong. HEAD requests are supposed to be identical to GET requests in every way _except_ that there is no message
                      Message 10 of 25 , Aug 2, 2004
                      • 0 Attachment
                        > But my point is, for a HEAD request, there is no data so apache should not
                        > touch my content-length header. I really dislike to generate the full data
                        > for the request and apache throws it away ( and even the I get no
                        > Content-Length header ).

                        for the record, this is fundamentally wrong. HEAD requests are supposed to
                        be identical to GET requests in every way _except_ that there is no message
                        body, which means that if a GET request for a specific resource does not
                        have a C-L header then a HEAD request for the same resource also _must_ not
                        have a C-L header. at least if you care about RFC compliance.

                        --Geoff

                        --
                        Report problems: http://perl.apache.org/bugs/
                        Mail list info: http://perl.apache.org/maillist/modperl.html
                        List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                      • Boris Zentner
                        Hi, ... ??? I do not know what you mean, my GET request _has_ a content-lenght header! For HEAD, I just do not calculate the expencive data for my body. I just
                        Message 11 of 25 , Aug 2, 2004
                        • 0 Attachment
                          Hi,

                          Am Montag 02 August 2004 15:28 schrieb Geoffrey Young:
                          > > But my point is, for a HEAD request, there is no data so apache should
                          > > not touch my content-length header. I really dislike to generate the full
                          > > data for the request and apache throws it away ( and even the I get no
                          > > Content-Length header ).
                          >
                          > for the record, this is fundamentally wrong. HEAD requests are supposed to
                          > be identical to GET requests in every way _except_ that there is no message
                          > body, which means that if a GET request for a specific resource does not
                          > have a C-L header then a HEAD request for the same resource also _must_ not
                          > have a C-L header. at least if you care about RFC compliance.
                          >

                          ??? I do not know what you mean, my GET request _has_ a content-lenght header!
                          For HEAD, I just do not calculate the expencive data for my body.

                          I just missed the content-length header on HEAD requests that is delivered on
                          GET.

                          > --Geoff

                          --
                          Boris

                          --
                          Report problems: http://perl.apache.org/bugs/
                          Mail list info: http://perl.apache.org/maillist/modperl.html
                          List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                        • Stas Bekman
                          ... Geoff, see the t/apache/head_request.t test I ve added last night. If you don t send a body, Apache strips the C-L header for HEAD requests. Sending at
                          Message 12 of 25 , Aug 2, 2004
                          • 0 Attachment
                            Boris Zentner wrote:
                            > Hi,
                            >
                            > Am Montag 02 August 2004 15:28 schrieb Geoffrey Young:
                            >
                            >>>But my point is, for a HEAD request, there is no data so apache should
                            >>>not touch my content-length header. I really dislike to generate the full
                            >>>data for the request and apache throws it away ( and even the I get no
                            >>>Content-Length header ).
                            >>
                            >>for the record, this is fundamentally wrong. HEAD requests are supposed to
                            >>be identical to GET requests in every way _except_ that there is no message
                            >>body, which means that if a GET request for a specific resource does not
                            >>have a C-L header then a HEAD request for the same resource also _must_ not
                            >>have a C-L header. at least if you care about RFC compliance.
                            >>
                            >
                            >
                            > ??? I do not know what you mean, my GET request _has_ a content-lenght header!
                            > For HEAD, I just do not calculate the expencive data for my body.
                            >
                            > I just missed the content-length header on HEAD requests that is delivered on
                            > GET.

                            Geoff, see the t/apache/head_request.t test I've added last night. If
                            you don't send a body, Apache strips the C-L header for HEAD requests.
                            Sending at least 1 byte works as a workaround, but it smells like a bug
                            in Apache.


                            --
                            __________________________________________________________________
                            Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                            http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                            mailto:stas@... http://use.perl.org http://apacheweek.com
                            http://modperlbook.org http://apache.org http://ticketmaster.com

                            --
                            Report problems: http://perl.apache.org/bugs/
                            Mail list info: http://perl.apache.org/maillist/modperl.html
                            List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                          • Geoffrey Young
                            ... that is exactly what I mean - if you include a C-L header on a GET then you are supposed to have one for a HEAD request as well, expensive or not. HEAD is
                            Message 13 of 25 , Aug 2, 2004
                            • 0 Attachment
                              >> ??? I do not know what you mean, my GET request _has_ a content-lenght
                              >> header! For HEAD, I just do not calculate the expencive data for my body.

                              that is exactly what I mean - if you include a C-L header on a GET then you
                              are supposed to have one for a HEAD request as well, expensive or not. HEAD
                              is supposed to be exactly the same as a GET in all respects except that it
                              does not have a message body.

                              >>
                              >> I just missed the content-length header on HEAD requests that is
                              >> delivered on GET.
                              >
                              >
                              > Geoff, see the t/apache/head_request.t test I've added last night. If
                              > you don't send a body, Apache strips the C-L header for HEAD requests.
                              > Sending at least 1 byte works as a workaround, but it smells like a bug
                              > in Apache.

                              yes, clearly there is a bug in there, but not where you think - what is
                              important is that apache needs to do the same thing on GET as HEAD, not that
                              there is no C-L generated for contentless HEAD requests. this patch more
                              accurately represents the real bug which, IIRC, httpd is already aware of
                              (at least I'm recalling discussion about a C-L of 0).

                              --Geoff
                            • Jean-Michel Hiver
                              ... From RFC 2616: The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response. The metainformation contained in
                              Message 14 of 25 , Aug 2, 2004
                              • 0 Attachment
                                Geoffrey Young wrote:
                                >>>??? I do not know what you mean, my GET request _has_ a content-lenght
                                >>>header! For HEAD, I just do not calculate the expencive data for my body.
                                >
                                > that is exactly what I mean - if you include a C-L header on a GET then you
                                > are supposed to have one for a HEAD request as well, expensive or not. HEAD
                                > is supposed to be exactly the same as a GET in all respects except that it
                                > does not have a message body.


                                From RFC 2616:

                                "The HEAD method is identical to GET except that the server MUST NOT
                                return a message-body in the response. The metainformation contained in
                                the HTTP headers in response to a HEAD request SHOULD be identical to
                                the information sent in response to a GET request."


                                From RFC 2119:

                                "SHOULD This word, or the adjective "RECOMMENDED", mean that there
                                may exist valid reasons in particular circumstances to ignore a
                                particular item, but the full implications must be understood and
                                carefully weighed before choosing a different course."


                                I think here we've hit a "valid reason in particular circumstance" to
                                strip the content-length header.

                                As for the implications, content-length is used for persistent HTTP
                                connections but that doesn't happen for HEAD requests - only for GETs.
                                Hence there should be no ill side effects by stripping the
                                Content-Length metadata for HEAD requests.


                                Cheers,
                                Jean-Michel.

                                --
                                Report problems: http://perl.apache.org/bugs/
                                Mail list info: http://perl.apache.org/maillist/modperl.html
                                List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                              • Geoffrey Young
                                ... indeed :) ... yeah, this is the conversation that comes up all the time. from what I ve seen in various forums, despite this definition SHOULD essentially
                                Message 15 of 25 , Aug 2, 2004
                                • 0 Attachment
                                  > From RFC 2616:
                                  >
                                  > "The HEAD method is identical to GET except that the server MUST NOT
                                  > return a message-body in the response. The metainformation contained in
                                  > the HTTP headers in response to a HEAD request SHOULD be identical to
                                  > the information sent in response to a GET request."

                                  indeed :)

                                  >
                                  >
                                  > From RFC 2119:
                                  >
                                  > "SHOULD This word, or the adjective "RECOMMENDED", mean that there
                                  > may exist valid reasons in particular circumstances to ignore a
                                  > particular item, but the full implications must be understood and
                                  > carefully weighed before choosing a different course."

                                  yeah, this is the conversation that comes up all the time. from what I've
                                  seen in various forums, despite this definition SHOULD essentially means
                                  MUST. at least I haven't come across a circumstance where it hasn't :)

                                  > I think here we've hit a "valid reason in particular circumstance" to
                                  > strip the content-length header.

                                  well, that depends. but the short of it is that almost nobody is HEAD
                                  compliant (see a recent thread in the past month wrt mozilla and HEAD and
                                  yahoo). and almost nobody is compliant because almost nobody understands
                                  exactly what a HEAD request is supposed to entail, thus the reason why I was
                                  belaboring the point. but if you go against the RFC "recommendation"
                                  knowing full well what you are doing, I guess that's ok ;)

                                  >
                                  > As for the implications, content-length is used for persistent HTTP
                                  > connections but that doesn't happen for HEAD requests - only for GETs.
                                  > Hence there should be no ill side effects by stripping the
                                  > Content-Length metadata for HEAD requests.

                                  actually, I wonder how true that is. you would certainly think so, but if
                                  you go back to that mozilla thread and read through the bugzilla entries the
                                  mozilla dev guys make it sound as though they are using a HEAD request then
                                  making a series of GETs over the same connection. but what do I know...

                                  --Geoff

                                  --
                                  Report problems: http://perl.apache.org/bugs/
                                  Mail list info: http://perl.apache.org/maillist/modperl.html
                                  List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                • Stas Bekman
                                  Geoffrey Young wrote: [...] ... Why do you say so? Your patch respresents a related, but a different issue. Boris has a problem with HEAD, and that s what the
                                  Message 16 of 25 , Aug 2, 2004
                                  • 0 Attachment
                                    Geoffrey Young wrote:
                                    [...]
                                    >>>I just missed the content-length header on HEAD requests that is
                                    >>>delivered on GET.
                                    >>
                                    >>
                                    >>Geoff, see the t/apache/head_request.t test I've added last night. If
                                    >>you don't send a body, Apache strips the C-L header for HEAD requests.
                                    >>Sending at least 1 byte works as a workaround, but it smells like a bug
                                    >>in Apache.
                                    >
                                    >
                                    > yes, clearly there is a bug in there, but not where you think - what is
                                    > important is that apache needs to do the same thing on GET as HEAD, not that
                                    > there is no C-L generated for contentless HEAD requests. this patch more
                                    > accurately represents the real bug which, IIRC, httpd is already aware of
                                    > (at least I'm recalling discussion about a C-L of 0).

                                    Why do you say so? Your patch respresents a related, but a different
                                    issue. Boris has a problem with HEAD, and that's what the test was testing.

                                    But, certainly please commit it, rename the files to something different
                                    (content_length_header.t?) since it's not longer a HEAD request. And
                                    please add a comment explaining the bug, ideally with xref to a bugzilla
                                    for the bug you are talking about or discussion thread.

                                    Thanks.

                                    also please move that no strict thing, inside the block, where it is
                                    needed. Thanks.

                                    > +no strict qw(refs);
                                    > +foreach my $method qw(GET HEAD) {

                                    --
                                    __________________________________________________________________
                                    Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                                    http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                                    mailto:stas@... http://use.perl.org http://apacheweek.com
                                    http://modperlbook.org http://apache.org http://ticketmaster.com

                                    --
                                    Report problems: http://perl.apache.org/bugs/
                                    Mail list info: http://perl.apache.org/maillist/modperl.html
                                    List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                  • Geoffrey Young
                                    ... ok, maybe I was misunderstanding the discussion and subsequent test. my points are these: - apache has essentially full reign over the C-L header - it
                                    Message 17 of 25 , Aug 2, 2004
                                    • 0 Attachment
                                      >> yes, clearly there is a bug in there, but not where you think - what is
                                      >> important is that apache needs to do the same thing on GET as HEAD,
                                      >> not that
                                      >> there is no C-L generated for contentless HEAD requests. this patch more
                                      >> accurately represents the real bug which, IIRC, httpd is already aware of
                                      >> (at least I'm recalling discussion about a C-L of 0).
                                      >
                                      >
                                      > Why do you say so? Your patch respresents a related, but a different
                                      > issue. Boris has a problem with HEAD, and that's what the test was testing.

                                      ok, maybe I was misunderstanding the discussion and subsequent test. my
                                      points are these:

                                      - apache has essentially full reign over the C-L header - it decides
                                      whether it is required or not, regardless of whether or not you call
                                      ap_set_content_length

                                      - more important than whether the C-L header shows up or not for a given
                                      HEAD or GET request is whether apache is consistent with itself. that is,
                                      regardless of what _you_ think it should do, apache _knows_ what it should
                                      do and should do it consistently between GET and HEAD.

                                      - of course, printing a C-L header to stdout works around all of this -
                                      apache can only manage things when you use the official API, since it checks
                                      the headers_out table and not the actual content.

                                      so, the test, as it existed before, made me think that the problem you saw
                                      was in that final test, that adding content magically made the C-L header
                                      appear. my point was that as long as apache does whatever it does
                                      consistently between GET and HEAD it's not a bug. but maybe I'm just not
                                      seeing it, so if you guys grok all of that and still see an issue, let's
                                      flesh it out.

                                      >
                                      > But, certainly please commit it, rename the files to something different
                                      > (content_length_header.t?) since it's not longer a HEAD request. And
                                      > please add a comment explaining the bug, ideally with xref to a bugzilla
                                      > for the bug you are talking about or discussion thread.

                                      will do.

                                      > also please move that no strict thing, inside the block, where it is
                                      > needed. Thanks.
                                      >
                                      >> +no strict qw(refs);
                                      >> +foreach my $method qw(GET HEAD) {

                                      d'oh!

                                      --Geoff

                                      --
                                      Report problems: http://perl.apache.org/bugs/
                                      Mail list info: http://perl.apache.org/maillist/modperl.html
                                      List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                    • Stas Bekman
                                      ... See my explanation of what I think is the problem at the end of this email. ... eh? how exactly do you do that? without assbackwards ofcourse. ... Actually
                                      Message 18 of 25 , Aug 2, 2004
                                      • 0 Attachment
                                        Geoffrey Young wrote:
                                        >>>yes, clearly there is a bug in there, but not where you think - what is
                                        >>>important is that apache needs to do the same thing on GET as HEAD,
                                        >>>not that
                                        >>>there is no C-L generated for contentless HEAD requests. this patch more
                                        >>>accurately represents the real bug which, IIRC, httpd is already aware of
                                        >>>(at least I'm recalling discussion about a C-L of 0).
                                        >>
                                        >>
                                        >>Why do you say so? Your patch respresents a related, but a different
                                        >>issue. Boris has a problem with HEAD, and that's what the test was testing.
                                        >
                                        >
                                        > ok, maybe I was misunderstanding the discussion and subsequent test. my
                                        > points are these:
                                        >
                                        > - apache has essentially full reign over the C-L header - it decides
                                        > whether it is required or not, regardless of whether or not you call
                                        > ap_set_content_length
                                        >
                                        > - more important than whether the C-L header shows up or not for a given
                                        > HEAD or GET request is whether apache is consistent with itself. that is,
                                        > regardless of what _you_ think it should do, apache _knows_ what it should
                                        > do and should do it consistently between GET and HEAD.

                                        See my explanation of what I think is the problem at the end of this email.

                                        > - of course, printing a C-L header to stdout works around all of this -
                                        > apache can only manage things when you use the official API, since it checks
                                        > the headers_out table and not the actual content.

                                        eh? how exactly do you do that? without assbackwards ofcourse.

                                        > so, the test, as it existed before, made me think that the problem you saw
                                        > was in that final test, that adding content magically made the C-L header
                                        > appear.

                                        Actually I the problem I saw was exactly what Boris was talking about:
                                        The C-L header wasn't there. The test simply exercises 4 different
                                        combinations of sending and not sending C-L header and content.

                                        > my point was that as long as apache does whatever it does
                                        > consistently between GET and HEAD it's not a bug. but maybe I'm just not
                                        > seeing it, so if you guys grok all of that and still see an issue, let's
                                        > flesh it out.

                                        I think it's there. The test that shows that is:

                                        {
                                        # if the response handler sends no data, and sets C-L header,
                                        # the client doesn't get C-L header
                                        my $uri = "$location?set_content_length";
                                        my $res = $method->($uri);
                                        ok t_cmp $res->code, 200, "$method $uri code";
                                        ok t_cmp $res->header('Content-Length'), undef, "$method $uri
                                        C-L header";
                                        ok t_cmp $res->content, "", "$method $uri content";
                                        }

                                        As I didn't know what's the right thing that should happen, I didn't
                                        make it a failure. But it probabaly shouldn't be undef in the C-L check.

                                        But the fact that sending a bogus char and a different C-L header works
                                        with HEAD (gets the C-L header through), whereas not sending any output
                                        breaks, seems like a bug to me. As mentioned earlier I think the bug is
                                        in the headers_out filter, which received EOS (since on HEAD apache
                                        scratches the response body) and no body. So it takes the liberty to
                                        nuke the C-L header, which I'm not sure is a good thing. When we send
                                        some body, headers_out sends the headers before seeing EOS and therefore
                                        it can't tell whether the body is coming or not, so it leaves the C-L
                                        header alone. Which is at least inconsistent.

                                        --
                                        __________________________________________________________________
                                        Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                                        http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                                        mailto:stas@... http://use.perl.org http://apacheweek.com
                                        http://modperlbook.org http://apache.org http://ticketmaster.com

                                        --
                                        Report problems: http://perl.apache.org/bugs/
                                        Mail list info: http://perl.apache.org/maillist/modperl.html
                                        List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                      • Geoffrey Young
                                        ... ok, I think I see what is going on, but it s different than what you see :) first, apache calls the C-L filter, which has no regard for any existing C-L
                                        Message 19 of 25 , Aug 2, 2004
                                        • 0 Attachment
                                          > As mentioned earlier I think the bug is
                                          > in the headers_out filter, which received EOS (since on HEAD apache
                                          > scratches the response body) and no body. So it takes the liberty to
                                          > nuke the C-L header, which I'm not sure is a good thing. When we send
                                          > some body, headers_out sends the headers before seeing EOS and therefore
                                          > it can't tell whether the body is coming or not, so it leaves the C-L
                                          > header alone. Which is at least inconsistent.

                                          ok, I think I see what is going on, but it's different than what you see :)

                                          first, apache calls the C-L filter, which has no regard for any existing C-L
                                          header and calculates its own if it can. so, in your test case the call to
                                          $r->set_content_length(25) is overwritten by the real C-L of zero, as
                                          calculated by the C-L filter.

                                          next the header filter kicks in. inside the header filter is this bit of logic:

                                          if (r->header_only
                                          && (clheader = apr_table_get(r->headers_out, "Content-Length"))
                                          && !strcmp(clheader, "0")) {
                                          apr_table_unset(r->headers_out, "Content-Length");
                                          }

                                          which treats HEAD requests with a C-L of zero as a special case. and, in
                                          fact, that is what you have sent it, at least in the test case. this is why
                                          GET and HEAD differ in this instance, because you haven't sent any real
                                          content and a C-L of 0 is special.

                                          so, I don't see the bug. well, I do see it, except that it is intentionally
                                          coded to be as such, which turns the bug into a feature.

                                          --Geoff

                                          --
                                          Report problems: http://perl.apache.org/bugs/
                                          Mail list info: http://perl.apache.org/maillist/modperl.html
                                          List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                        • Glenn Strauss
                                          On Mon, Aug 02, 2004 at 11:55:56AM -0700, Stas Bekman wrote: [...] ... I mentioned to Geoff off-list about possibly using a flush bucket, since I don t have a
                                          Message 20 of 25 , Aug 3, 2004
                                          • 0 Attachment
                                            On Mon, Aug 02, 2004 at 11:55:56AM -0700, Stas Bekman wrote:
                                            [...]
                                            > Actually I the problem I saw was exactly what Boris was talking about:
                                            > The C-L header wasn't there. The test simply exercises 4 different
                                            > combinations of sending and not sending C-L header and content.
                                            >
                                            > >my point was that as long as apache does whatever it does
                                            > >consistently between GET and HEAD it's not a bug. but maybe I'm just not
                                            > >seeing it, so if you guys grok all of that and still see an issue, let's
                                            > >flesh it out.
                                            >
                                            > I think it's there. The test that shows that is:
                                            >
                                            > {
                                            > # if the response handler sends no data, and sets C-L header,
                                            > # the client doesn't get C-L header
                                            > my $uri = "$location?set_content_length";
                                            > my $res = $method->($uri);
                                            > ok t_cmp $res->code, 200, "$method $uri code";
                                            > ok t_cmp $res->header('Content-Length'), undef, "$method $uri
                                            > C-L header";
                                            > ok t_cmp $res->content, "", "$method $uri content";
                                            > }
                                            >
                                            > As I didn't know what's the right thing that should happen, I didn't
                                            > make it a failure. But it probabaly shouldn't be undef in the C-L check.
                                            >
                                            > But the fact that sending a bogus char and a different C-L header works
                                            > with HEAD (gets the C-L header through), whereas not sending any output
                                            > breaks, seems like a bug to me. As mentioned earlier I think the bug is
                                            > in the headers_out filter, which received EOS (since on HEAD apache
                                            > scratches the response body) and no body. So it takes the liberty to
                                            > nuke the C-L header, which I'm not sure is a good thing. When we send
                                            > some body, headers_out sends the headers before seeing EOS and therefore
                                            > it can't tell whether the body is coming or not, so it leaves the C-L
                                            > header alone. Which is at least inconsistent.


                                            I mentioned to Geoff off-list about possibly using a flush bucket,
                                            since I don't have a test setup ready (just replaced my dead
                                            laptop hard drive)

                                            I wrote:
                                            > > I used to make good use of $r->send_http_header() in my MP1 handlers.
                                            > > This method isn't available in MP2 (why?), though I could picture it
                                            > > replaced by sending a flush bucket down the filter chain.
                                            >
                                            On Tue, Aug 03, 2004 at 10:04:38AM -0400, Geoffrey Young wrote:
                                            > apache now sends the headers for you via the header filter. if you were
                                            > allowed to send your own headers you might send them before a filter ran
                                            > that would alter the headers. for instance, mod_include removes the ETag
                                            > header.

                                            Would a flush bucket be a great way to say "send headers now"?
                                            So if you're not going to send the actual content down the filter chain
                                            on a HEAD request, would sending a flush bucket make things happy?

                                            IIF this works, can I make a request the $r->send_http_header() be
                                            implemented in Apache2 to do just that? (create and pass a flush bucket)

                                            Cheers,
                                            Glenn

                                            --
                                            Report problems: http://perl.apache.org/bugs/
                                            Mail list info: http://perl.apache.org/maillist/modperl.html
                                            List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                          • Stas Bekman
                                            ... Thanks Glenn, calling $r- rflush works perfectly fine to cause user-preset C-L header to be delivered to the client on HEAD request. ... I think this is
                                            Message 21 of 25 , Aug 4, 2004
                                            • 0 Attachment
                                              Glenn Strauss wrote:
                                              > On Mon, Aug 02, 2004 at 11:55:56AM -0700, Stas Bekman wrote:
                                              > [...]

                                              > I mentioned to Geoff off-list about possibly using a flush bucket,
                                              > since I don't have a test setup ready (just replaced my dead
                                              > laptop hard drive)

                                              Thanks Glenn, calling $r->rflush works perfectly fine to cause
                                              user-preset C-L header to be delivered to the client on HEAD request.

                                              > I wrote:
                                              >
                                              >>>I used to make good use of $r->send_http_header() in my MP1 handlers.
                                              >>>This method isn't available in MP2 (why?), though I could picture it
                                              >>>replaced by sending a flush bucket down the filter chain.
                                              >>
                                              > On Tue, Aug 03, 2004 at 10:04:38AM -0400, Geoffrey Young wrote:
                                              >
                                              >>apache now sends the headers for you via the header filter. if you were
                                              >>allowed to send your own headers you might send them before a filter ran
                                              >>that would alter the headers. for instance, mod_include removes the ETag
                                              >>header.

                                              I think this is not the reason why ap_send_http_header was removed from
                                              Apache2 API. It's simply not needed because the headers_out filter will
                                              inject the headers as soon as some body is sent. Your example of
                                              mod_include doesn't work if the response handler does:

                                              $r->print("foo bar");
                                              $r->rflush; # or if the previous print was big enough to cause flush
                                              $r->print(q[<!--#include virtual="/foo" -->]);

                                              > Would a flush bucket be a great way to say "send headers now"?
                                              > So if you're not going to send the actual content down the filter chain
                                              > on a HEAD request, would sending a flush bucket make things happy?

                                              It does.

                                              > IIF this works, can I make a request the $r->send_http_header() be
                                              > implemented in Apache2 to do just that? (create and pass a flush bucket)

                                              Absolutely. But it'd be nice if you could first ask the httpd-dev list:
                                              http://httpd.apache.org/lists.html#http-dev
                                              if they are interested in adding the C API to do that.

                                              If httpd-dev doesn't do it, I suppose we can still implement it in mp2.

                                              --
                                              __________________________________________________________________
                                              Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                                              http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                                              mailto:stas@... http://use.perl.org http://apacheweek.com
                                              http://modperlbook.org http://apache.org http://ticketmaster.com

                                              --
                                              Report problems: http://perl.apache.org/bugs/
                                              Mail list info: http://perl.apache.org/maillist/modperl.html
                                              List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                            • Stas Bekman
                                              Stas Bekman wrote: [...] ... On a second thought, I think it s better not to add this special method unless Apache does that. Since in 99.99% cases one should
                                              Message 22 of 25 , Aug 4, 2004
                                              • 0 Attachment
                                                Stas Bekman wrote:
                                                [...]
                                                >> Would a flush bucket be a great way to say "send headers now"?
                                                >> So if you're not going to send the actual content down the filter chain
                                                >> on a HEAD request, would sending a flush bucket make things happy?
                                                >
                                                >
                                                > It does.
                                                >
                                                >> IIF this works, can I make a request the $r->send_http_header() be
                                                >> implemented in Apache2 to do just that? (create and pass a flush bucket)
                                                >
                                                >
                                                > Absolutely. But it'd be nice if you could first ask the httpd-dev list:
                                                > http://httpd.apache.org/lists.html#http-dev
                                                > if they are interested in adding the C API to do that.
                                                >
                                                > If httpd-dev doesn't do it, I suppose we can still implement it in mp2.

                                                On a second thought, I think it's better not to add this special method
                                                unless Apache does that. Since in 99.99% cases one should not cause an
                                                early headers sending, and for those special cases which go against the
                                                Apache spec (sending no body for HEAD requests) we will simply document
                                                a workaround. This is because the workaround may stop working in the
                                                future and we will find ourselves in a bad situation.

                                                If Apache adds this method and documents it usage for the purposes
                                                similar to one discussed in this thread, then Apache will need to ensure
                                                that it'll work in the future. Therefore if you think send_http_header()
                                                is needed, please discuss that on the quoted above list, and depending
                                                on the outcome we will either provide the perl interface to it or simply
                                                document a workaround, which may or may not stop working in the future.

                                                --
                                                __________________________________________________________________
                                                Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                                                http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                                                mailto:stas@... http://use.perl.org http://apacheweek.com
                                                http://modperlbook.org http://apache.org http://ticketmaster.com

                                                --
                                                Report problems: http://perl.apache.org/bugs/
                                                Mail list info: http://perl.apache.org/maillist/modperl.html
                                                List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                              • Stas Bekman
                                                ... Thanks Geoff for looking at the source code and giving us the full story. I ve now converted it into the comments in the t/apache/content_length_header.t
                                                Message 23 of 25 , Aug 4, 2004
                                                • 0 Attachment
                                                  Geoffrey Young wrote:
                                                  >
                                                  >>As mentioned earlier I think the bug is
                                                  >>in the headers_out filter, which received EOS (since on HEAD apache
                                                  >>scratches the response body) and no body. So it takes the liberty to
                                                  >>nuke the C-L header, which I'm not sure is a good thing. When we send
                                                  >>some body, headers_out sends the headers before seeing EOS and therefore
                                                  >>it can't tell whether the body is coming or not, so it leaves the C-L
                                                  >>header alone. Which is at least inconsistent.
                                                  >
                                                  >
                                                  > ok, I think I see what is going on, but it's different than what you see :)
                                                  >
                                                  > first, apache calls the C-L filter, which has no regard for any existing C-L
                                                  > header and calculates its own if it can. so, in your test case the call to
                                                  > $r->set_content_length(25) is overwritten by the real C-L of zero, as
                                                  > calculated by the C-L filter.
                                                  >
                                                  > next the header filter kicks in. inside the header filter is this bit of logic:
                                                  >
                                                  > if (r->header_only
                                                  > && (clheader = apr_table_get(r->headers_out, "Content-Length"))
                                                  > && !strcmp(clheader, "0")) {
                                                  > apr_table_unset(r->headers_out, "Content-Length");
                                                  > }
                                                  >
                                                  > which treats HEAD requests with a C-L of zero as a special case. and, in
                                                  > fact, that is what you have sent it, at least in the test case. this is why
                                                  > GET and HEAD differ in this instance, because you haven't sent any real
                                                  > content and a C-L of 0 is special.
                                                  >
                                                  > so, I don't see the bug. well, I do see it, except that it is intentionally
                                                  > coded to be as such, which turns the bug into a feature.

                                                  Thanks Geoff for looking at the source code and giving us the full
                                                  story. I've now converted it into the comments in the
                                                  t/apache/content_length_header.t test and I also used Glenn's suggestion
                                                  to use $r->rflush, which gives us:

                                                  # 2. even though the spec says that content handlers should send an
                                                  # identical response for GET and HEAD requests, some folks try to
                                                  # avoid the overhead of generating the response body, which Apache is
                                                  # going to discard anyway for HEAD requests. The following discussion
                                                  # assumes that we deal with a HEAD request.
                                                  #
                                                  # When Apache sees EOS and no headers and no response body were sent,
                                                  # ap_content_length_filter (httpd-2.0/server/protocol.c) sets C-L to
                                                  # 0. Later on ap_http_header_filter
                                                  # (httpd-2.0/modules/http/http_protocol.c) removes the C-L header for
                                                  # the HEAD requests
                                                  #
                                                  # the workaround is to force the sending of the response headers,
                                                  # before EOS was sent. The simplest solution is to use rflush():
                                                  #
                                                  # if ($r->header_only) { # HEAD
                                                  # $body_len = calculate_body_len();
                                                  # $r->set_content_length($body_len);
                                                  # $r->rflush;
                                                  # }
                                                  # else { # GET
                                                  # # generate and send the body
                                                  # }
                                                  #
                                                  # now if the handler sets the C-L header it'll be delivered to the
                                                  # client unmodified.


                                                  --
                                                  __________________________________________________________________
                                                  Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                                                  http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                                                  mailto:stas@... http://use.perl.org http://apacheweek.com
                                                  http://modperlbook.org http://apache.org http://ticketmaster.com

                                                  --
                                                  Report problems: http://perl.apache.org/bugs/
                                                  Mail list info: http://perl.apache.org/maillist/modperl.html
                                                  List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                                • Geoffrey Young
                                                  ... I fully agree. ... right - apache expects you to not do anything special for HEAD requests anymore, so if you mess with that contract you are messing with
                                                  Message 24 of 25 , Aug 4, 2004
                                                  • 0 Attachment
                                                    >> If httpd-dev doesn't do it, I suppose we can still implement it in mp2.
                                                    >
                                                    >
                                                    > On a second thought, I think it's better not to add this special method
                                                    > unless Apache does that.

                                                    I fully agree.

                                                    > Since in 99.99% cases one should not cause an
                                                    > early headers sending, and for those special cases which go against the
                                                    > Apache spec (sending no body for HEAD requests) we will simply document
                                                    > a workaround. This is because the workaround may stop working in the
                                                    > future and we will find ourselves in a bad situation.

                                                    right - apache expects you to not do anything special for HEAD requests
                                                    anymore, so if you mess with that contract you are messing with an
                                                    unsupported feature, and mod_perl has no business supporting unsupported
                                                    apache features through a new, mod_perl-specific API.

                                                    --Geoff

                                                    --
                                                    Report problems: http://perl.apache.org/bugs/
                                                    Mail list info: http://perl.apache.org/maillist/modperl.html
                                                    List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                                  • Stas Bekman
                                                    This thread has been documented at: http://perl.apache.org/docs/2.0/user/handlers/http.html#C_Content_Length__Response_Header --
                                                    Message 25 of 25 , Aug 8, 2004
                                                    • 0 Attachment
                                                      This thread has been documented at:
                                                      http://perl.apache.org/docs/2.0/user/handlers/http.html#C_Content_Length__Response_Header

                                                      --
                                                      __________________________________________________________________
                                                      Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                                                      http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                                                      mailto:stas@... http://use.perl.org http://apacheweek.com
                                                      http://modperlbook.org http://apache.org http://ticketmaster.com

                                                      --
                                                      Report problems: http://perl.apache.org/bugs/
                                                      Mail list info: http://perl.apache.org/maillist/modperl.html
                                                      List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                                    Your message has been successfully submitted and would be delivered to recipients shortly.