Loading ...
Sorry, an error occurred while loading the content.

Re: mod_perl2, HEAD request and Content-Length

Expand Messages
  • Stas Bekman
    ... I get indentical behavior with either of the two methods: my $body = This is a response string ; #$r- headers_out- { Content-Length } = length $body;
    Message 1 of 25 , Aug 1, 2004
    • 0 Attachment
      Stas Bekman wrote:

      > let me try to write a test and I'll be back to you.

      I get indentical behavior with either of the two methods:

      my $body = "This is a response string";
      #$r->headers_out->{'Content-Length'} = length $body;
      $r->set_content_length(length $body);

      $r->print('a');

      if the one-char-print line is commented out, I *never* get the C-L
      header for HEAD requests. Otherwise I always get the correct C-L header.

      Normally, Apache is not able to figure out how much data the content
      handler is going to send, since it needs to send headers early. However
      in the case where your handler returns w/o sending anything at all, the
      headers_out filter sees EOS and no data, so it probably thinks that C-L
      is wrong and drops it. My test with printing just one char makes it
      working. It's safe to send anything, since Apache will discard it for
      HEAD requests anyway. So here you have a workaround (but I can't promise
      that it will always continue to work).

      Please ask at the Apache list, whether there is a cleaner way to
      make Apache preserve the C-L header, w/o sending any data, and please
      let us know the verdict.

      p.s. I'm not sure how you've got a HEAD request with C-L header working,
      when using $r->headers_out->{'Content-Length'}; it doesn't work for me.
      (though I use Apache 2.0.50-dev so I won't be surprised if something has
      changed).

      --
      __________________________________________________________________
      Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
      http://stason.org/ mod_perl Guide ---> http://perl.apache.org
      mailto:stas@... http://use.perl.org http://apacheweek.com
      http://modperlbook.org http://apache.org http://ticketmaster.com

      --
      Report problems: http://perl.apache.org/bugs/
      Mail list info: http://perl.apache.org/maillist/modperl.html
      List etiquette: http://perl.apache.org/maillist/email-etiquette.html
    • Stas Bekman
      ... I ve just committed this test to the modperl-2.0 cvs rep: t/apache/head_request.t t/response/TestApache/head_request.pm please take a look --
      Message 2 of 25 , Aug 1, 2004
      • 0 Attachment
        Stas Bekman wrote:
        > Stas Bekman wrote:
        >
        >> let me try to write a test and I'll be back to you.

        I've just committed this test to the modperl-2.0 cvs rep:
        t/apache/head_request.t t/response/TestApache/head_request.pm
        please take a look

        --
        __________________________________________________________________
        Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
        http://stason.org/ mod_perl Guide ---> http://perl.apache.org
        mailto:stas@... http://use.perl.org http://apacheweek.com
        http://modperlbook.org http://apache.org http://ticketmaster.com

        --
        Report problems: http://perl.apache.org/bugs/
        Mail list info: http://perl.apache.org/maillist/modperl.html
        List etiquette: http://perl.apache.org/maillist/email-etiquette.html
      • Geoffrey Young
        ... for the record, this is fundamentally wrong. HEAD requests are supposed to be identical to GET requests in every way _except_ that there is no message
        Message 3 of 25 , Aug 2, 2004
        • 0 Attachment
          > But my point is, for a HEAD request, there is no data so apache should not
          > touch my content-length header. I really dislike to generate the full data
          > for the request and apache throws it away ( and even the I get no
          > Content-Length header ).

          for the record, this is fundamentally wrong. HEAD requests are supposed to
          be identical to GET requests in every way _except_ that there is no message
          body, which means that if a GET request for a specific resource does not
          have a C-L header then a HEAD request for the same resource also _must_ not
          have a C-L header. at least if you care about RFC compliance.

          --Geoff

          --
          Report problems: http://perl.apache.org/bugs/
          Mail list info: http://perl.apache.org/maillist/modperl.html
          List etiquette: http://perl.apache.org/maillist/email-etiquette.html
        • Boris Zentner
          Hi, ... ??? I do not know what you mean, my GET request _has_ a content-lenght header! For HEAD, I just do not calculate the expencive data for my body. I just
          Message 4 of 25 , Aug 2, 2004
          • 0 Attachment
            Hi,

            Am Montag 02 August 2004 15:28 schrieb Geoffrey Young:
            > > But my point is, for a HEAD request, there is no data so apache should
            > > not touch my content-length header. I really dislike to generate the full
            > > data for the request and apache throws it away ( and even the I get no
            > > Content-Length header ).
            >
            > for the record, this is fundamentally wrong. HEAD requests are supposed to
            > be identical to GET requests in every way _except_ that there is no message
            > body, which means that if a GET request for a specific resource does not
            > have a C-L header then a HEAD request for the same resource also _must_ not
            > have a C-L header. at least if you care about RFC compliance.
            >

            ??? I do not know what you mean, my GET request _has_ a content-lenght header!
            For HEAD, I just do not calculate the expencive data for my body.

            I just missed the content-length header on HEAD requests that is delivered on
            GET.

            > --Geoff

            --
            Boris

            --
            Report problems: http://perl.apache.org/bugs/
            Mail list info: http://perl.apache.org/maillist/modperl.html
            List etiquette: http://perl.apache.org/maillist/email-etiquette.html
          • Stas Bekman
            ... Geoff, see the t/apache/head_request.t test I ve added last night. If you don t send a body, Apache strips the C-L header for HEAD requests. Sending at
            Message 5 of 25 , Aug 2, 2004
            • 0 Attachment
              Boris Zentner wrote:
              > Hi,
              >
              > Am Montag 02 August 2004 15:28 schrieb Geoffrey Young:
              >
              >>>But my point is, for a HEAD request, there is no data so apache should
              >>>not touch my content-length header. I really dislike to generate the full
              >>>data for the request and apache throws it away ( and even the I get no
              >>>Content-Length header ).
              >>
              >>for the record, this is fundamentally wrong. HEAD requests are supposed to
              >>be identical to GET requests in every way _except_ that there is no message
              >>body, which means that if a GET request for a specific resource does not
              >>have a C-L header then a HEAD request for the same resource also _must_ not
              >>have a C-L header. at least if you care about RFC compliance.
              >>
              >
              >
              > ??? I do not know what you mean, my GET request _has_ a content-lenght header!
              > For HEAD, I just do not calculate the expencive data for my body.
              >
              > I just missed the content-length header on HEAD requests that is delivered on
              > GET.

              Geoff, see the t/apache/head_request.t test I've added last night. If
              you don't send a body, Apache strips the C-L header for HEAD requests.
              Sending at least 1 byte works as a workaround, but it smells like a bug
              in Apache.


              --
              __________________________________________________________________
              Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
              http://stason.org/ mod_perl Guide ---> http://perl.apache.org
              mailto:stas@... http://use.perl.org http://apacheweek.com
              http://modperlbook.org http://apache.org http://ticketmaster.com

              --
              Report problems: http://perl.apache.org/bugs/
              Mail list info: http://perl.apache.org/maillist/modperl.html
              List etiquette: http://perl.apache.org/maillist/email-etiquette.html
            • Geoffrey Young
              ... that is exactly what I mean - if you include a C-L header on a GET then you are supposed to have one for a HEAD request as well, expensive or not. HEAD is
              Message 6 of 25 , Aug 2, 2004
              • 0 Attachment
                >> ??? I do not know what you mean, my GET request _has_ a content-lenght
                >> header! For HEAD, I just do not calculate the expencive data for my body.

                that is exactly what I mean - if you include a C-L header on a GET then you
                are supposed to have one for a HEAD request as well, expensive or not. HEAD
                is supposed to be exactly the same as a GET in all respects except that it
                does not have a message body.

                >>
                >> I just missed the content-length header on HEAD requests that is
                >> delivered on GET.
                >
                >
                > Geoff, see the t/apache/head_request.t test I've added last night. If
                > you don't send a body, Apache strips the C-L header for HEAD requests.
                > Sending at least 1 byte works as a workaround, but it smells like a bug
                > in Apache.

                yes, clearly there is a bug in there, but not where you think - what is
                important is that apache needs to do the same thing on GET as HEAD, not that
                there is no C-L generated for contentless HEAD requests. this patch more
                accurately represents the real bug which, IIRC, httpd is already aware of
                (at least I'm recalling discussion about a C-L of 0).

                --Geoff
              • Jean-Michel Hiver
                ... From RFC 2616: The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response. The metainformation contained in
                Message 7 of 25 , Aug 2, 2004
                • 0 Attachment
                  Geoffrey Young wrote:
                  >>>??? I do not know what you mean, my GET request _has_ a content-lenght
                  >>>header! For HEAD, I just do not calculate the expencive data for my body.
                  >
                  > that is exactly what I mean - if you include a C-L header on a GET then you
                  > are supposed to have one for a HEAD request as well, expensive or not. HEAD
                  > is supposed to be exactly the same as a GET in all respects except that it
                  > does not have a message body.


                  From RFC 2616:

                  "The HEAD method is identical to GET except that the server MUST NOT
                  return a message-body in the response. The metainformation contained in
                  the HTTP headers in response to a HEAD request SHOULD be identical to
                  the information sent in response to a GET request."


                  From RFC 2119:

                  "SHOULD This word, or the adjective "RECOMMENDED", mean that there
                  may exist valid reasons in particular circumstances to ignore a
                  particular item, but the full implications must be understood and
                  carefully weighed before choosing a different course."


                  I think here we've hit a "valid reason in particular circumstance" to
                  strip the content-length header.

                  As for the implications, content-length is used for persistent HTTP
                  connections but that doesn't happen for HEAD requests - only for GETs.
                  Hence there should be no ill side effects by stripping the
                  Content-Length metadata for HEAD requests.


                  Cheers,
                  Jean-Michel.

                  --
                  Report problems: http://perl.apache.org/bugs/
                  Mail list info: http://perl.apache.org/maillist/modperl.html
                  List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                • Geoffrey Young
                  ... indeed :) ... yeah, this is the conversation that comes up all the time. from what I ve seen in various forums, despite this definition SHOULD essentially
                  Message 8 of 25 , Aug 2, 2004
                  • 0 Attachment
                    > From RFC 2616:
                    >
                    > "The HEAD method is identical to GET except that the server MUST NOT
                    > return a message-body in the response. The metainformation contained in
                    > the HTTP headers in response to a HEAD request SHOULD be identical to
                    > the information sent in response to a GET request."

                    indeed :)

                    >
                    >
                    > From RFC 2119:
                    >
                    > "SHOULD This word, or the adjective "RECOMMENDED", mean that there
                    > may exist valid reasons in particular circumstances to ignore a
                    > particular item, but the full implications must be understood and
                    > carefully weighed before choosing a different course."

                    yeah, this is the conversation that comes up all the time. from what I've
                    seen in various forums, despite this definition SHOULD essentially means
                    MUST. at least I haven't come across a circumstance where it hasn't :)

                    > I think here we've hit a "valid reason in particular circumstance" to
                    > strip the content-length header.

                    well, that depends. but the short of it is that almost nobody is HEAD
                    compliant (see a recent thread in the past month wrt mozilla and HEAD and
                    yahoo). and almost nobody is compliant because almost nobody understands
                    exactly what a HEAD request is supposed to entail, thus the reason why I was
                    belaboring the point. but if you go against the RFC "recommendation"
                    knowing full well what you are doing, I guess that's ok ;)

                    >
                    > As for the implications, content-length is used for persistent HTTP
                    > connections but that doesn't happen for HEAD requests - only for GETs.
                    > Hence there should be no ill side effects by stripping the
                    > Content-Length metadata for HEAD requests.

                    actually, I wonder how true that is. you would certainly think so, but if
                    you go back to that mozilla thread and read through the bugzilla entries the
                    mozilla dev guys make it sound as though they are using a HEAD request then
                    making a series of GETs over the same connection. but what do I know...

                    --Geoff

                    --
                    Report problems: http://perl.apache.org/bugs/
                    Mail list info: http://perl.apache.org/maillist/modperl.html
                    List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                  • Stas Bekman
                    Geoffrey Young wrote: [...] ... Why do you say so? Your patch respresents a related, but a different issue. Boris has a problem with HEAD, and that s what the
                    Message 9 of 25 , Aug 2, 2004
                    • 0 Attachment
                      Geoffrey Young wrote:
                      [...]
                      >>>I just missed the content-length header on HEAD requests that is
                      >>>delivered on GET.
                      >>
                      >>
                      >>Geoff, see the t/apache/head_request.t test I've added last night. If
                      >>you don't send a body, Apache strips the C-L header for HEAD requests.
                      >>Sending at least 1 byte works as a workaround, but it smells like a bug
                      >>in Apache.
                      >
                      >
                      > yes, clearly there is a bug in there, but not where you think - what is
                      > important is that apache needs to do the same thing on GET as HEAD, not that
                      > there is no C-L generated for contentless HEAD requests. this patch more
                      > accurately represents the real bug which, IIRC, httpd is already aware of
                      > (at least I'm recalling discussion about a C-L of 0).

                      Why do you say so? Your patch respresents a related, but a different
                      issue. Boris has a problem with HEAD, and that's what the test was testing.

                      But, certainly please commit it, rename the files to something different
                      (content_length_header.t?) since it's not longer a HEAD request. And
                      please add a comment explaining the bug, ideally with xref to a bugzilla
                      for the bug you are talking about or discussion thread.

                      Thanks.

                      also please move that no strict thing, inside the block, where it is
                      needed. Thanks.

                      > +no strict qw(refs);
                      > +foreach my $method qw(GET HEAD) {

                      --
                      __________________________________________________________________
                      Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                      http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                      mailto:stas@... http://use.perl.org http://apacheweek.com
                      http://modperlbook.org http://apache.org http://ticketmaster.com

                      --
                      Report problems: http://perl.apache.org/bugs/
                      Mail list info: http://perl.apache.org/maillist/modperl.html
                      List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                    • Geoffrey Young
                      ... ok, maybe I was misunderstanding the discussion and subsequent test. my points are these: - apache has essentially full reign over the C-L header - it
                      Message 10 of 25 , Aug 2, 2004
                      • 0 Attachment
                        >> yes, clearly there is a bug in there, but not where you think - what is
                        >> important is that apache needs to do the same thing on GET as HEAD,
                        >> not that
                        >> there is no C-L generated for contentless HEAD requests. this patch more
                        >> accurately represents the real bug which, IIRC, httpd is already aware of
                        >> (at least I'm recalling discussion about a C-L of 0).
                        >
                        >
                        > Why do you say so? Your patch respresents a related, but a different
                        > issue. Boris has a problem with HEAD, and that's what the test was testing.

                        ok, maybe I was misunderstanding the discussion and subsequent test. my
                        points are these:

                        - apache has essentially full reign over the C-L header - it decides
                        whether it is required or not, regardless of whether or not you call
                        ap_set_content_length

                        - more important than whether the C-L header shows up or not for a given
                        HEAD or GET request is whether apache is consistent with itself. that is,
                        regardless of what _you_ think it should do, apache _knows_ what it should
                        do and should do it consistently between GET and HEAD.

                        - of course, printing a C-L header to stdout works around all of this -
                        apache can only manage things when you use the official API, since it checks
                        the headers_out table and not the actual content.

                        so, the test, as it existed before, made me think that the problem you saw
                        was in that final test, that adding content magically made the C-L header
                        appear. my point was that as long as apache does whatever it does
                        consistently between GET and HEAD it's not a bug. but maybe I'm just not
                        seeing it, so if you guys grok all of that and still see an issue, let's
                        flesh it out.

                        >
                        > But, certainly please commit it, rename the files to something different
                        > (content_length_header.t?) since it's not longer a HEAD request. And
                        > please add a comment explaining the bug, ideally with xref to a bugzilla
                        > for the bug you are talking about or discussion thread.

                        will do.

                        > also please move that no strict thing, inside the block, where it is
                        > needed. Thanks.
                        >
                        >> +no strict qw(refs);
                        >> +foreach my $method qw(GET HEAD) {

                        d'oh!

                        --Geoff

                        --
                        Report problems: http://perl.apache.org/bugs/
                        Mail list info: http://perl.apache.org/maillist/modperl.html
                        List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                      • Stas Bekman
                        ... See my explanation of what I think is the problem at the end of this email. ... eh? how exactly do you do that? without assbackwards ofcourse. ... Actually
                        Message 11 of 25 , Aug 2, 2004
                        • 0 Attachment
                          Geoffrey Young wrote:
                          >>>yes, clearly there is a bug in there, but not where you think - what is
                          >>>important is that apache needs to do the same thing on GET as HEAD,
                          >>>not that
                          >>>there is no C-L generated for contentless HEAD requests. this patch more
                          >>>accurately represents the real bug which, IIRC, httpd is already aware of
                          >>>(at least I'm recalling discussion about a C-L of 0).
                          >>
                          >>
                          >>Why do you say so? Your patch respresents a related, but a different
                          >>issue. Boris has a problem with HEAD, and that's what the test was testing.
                          >
                          >
                          > ok, maybe I was misunderstanding the discussion and subsequent test. my
                          > points are these:
                          >
                          > - apache has essentially full reign over the C-L header - it decides
                          > whether it is required or not, regardless of whether or not you call
                          > ap_set_content_length
                          >
                          > - more important than whether the C-L header shows up or not for a given
                          > HEAD or GET request is whether apache is consistent with itself. that is,
                          > regardless of what _you_ think it should do, apache _knows_ what it should
                          > do and should do it consistently between GET and HEAD.

                          See my explanation of what I think is the problem at the end of this email.

                          > - of course, printing a C-L header to stdout works around all of this -
                          > apache can only manage things when you use the official API, since it checks
                          > the headers_out table and not the actual content.

                          eh? how exactly do you do that? without assbackwards ofcourse.

                          > so, the test, as it existed before, made me think that the problem you saw
                          > was in that final test, that adding content magically made the C-L header
                          > appear.

                          Actually I the problem I saw was exactly what Boris was talking about:
                          The C-L header wasn't there. The test simply exercises 4 different
                          combinations of sending and not sending C-L header and content.

                          > my point was that as long as apache does whatever it does
                          > consistently between GET and HEAD it's not a bug. but maybe I'm just not
                          > seeing it, so if you guys grok all of that and still see an issue, let's
                          > flesh it out.

                          I think it's there. The test that shows that is:

                          {
                          # if the response handler sends no data, and sets C-L header,
                          # the client doesn't get C-L header
                          my $uri = "$location?set_content_length";
                          my $res = $method->($uri);
                          ok t_cmp $res->code, 200, "$method $uri code";
                          ok t_cmp $res->header('Content-Length'), undef, "$method $uri
                          C-L header";
                          ok t_cmp $res->content, "", "$method $uri content";
                          }

                          As I didn't know what's the right thing that should happen, I didn't
                          make it a failure. But it probabaly shouldn't be undef in the C-L check.

                          But the fact that sending a bogus char and a different C-L header works
                          with HEAD (gets the C-L header through), whereas not sending any output
                          breaks, seems like a bug to me. As mentioned earlier I think the bug is
                          in the headers_out filter, which received EOS (since on HEAD apache
                          scratches the response body) and no body. So it takes the liberty to
                          nuke the C-L header, which I'm not sure is a good thing. When we send
                          some body, headers_out sends the headers before seeing EOS and therefore
                          it can't tell whether the body is coming or not, so it leaves the C-L
                          header alone. Which is at least inconsistent.

                          --
                          __________________________________________________________________
                          Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                          http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                          mailto:stas@... http://use.perl.org http://apacheweek.com
                          http://modperlbook.org http://apache.org http://ticketmaster.com

                          --
                          Report problems: http://perl.apache.org/bugs/
                          Mail list info: http://perl.apache.org/maillist/modperl.html
                          List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                        • Geoffrey Young
                          ... ok, I think I see what is going on, but it s different than what you see :) first, apache calls the C-L filter, which has no regard for any existing C-L
                          Message 12 of 25 , Aug 2, 2004
                          • 0 Attachment
                            > As mentioned earlier I think the bug is
                            > in the headers_out filter, which received EOS (since on HEAD apache
                            > scratches the response body) and no body. So it takes the liberty to
                            > nuke the C-L header, which I'm not sure is a good thing. When we send
                            > some body, headers_out sends the headers before seeing EOS and therefore
                            > it can't tell whether the body is coming or not, so it leaves the C-L
                            > header alone. Which is at least inconsistent.

                            ok, I think I see what is going on, but it's different than what you see :)

                            first, apache calls the C-L filter, which has no regard for any existing C-L
                            header and calculates its own if it can. so, in your test case the call to
                            $r->set_content_length(25) is overwritten by the real C-L of zero, as
                            calculated by the C-L filter.

                            next the header filter kicks in. inside the header filter is this bit of logic:

                            if (r->header_only
                            && (clheader = apr_table_get(r->headers_out, "Content-Length"))
                            && !strcmp(clheader, "0")) {
                            apr_table_unset(r->headers_out, "Content-Length");
                            }

                            which treats HEAD requests with a C-L of zero as a special case. and, in
                            fact, that is what you have sent it, at least in the test case. this is why
                            GET and HEAD differ in this instance, because you haven't sent any real
                            content and a C-L of 0 is special.

                            so, I don't see the bug. well, I do see it, except that it is intentionally
                            coded to be as such, which turns the bug into a feature.

                            --Geoff

                            --
                            Report problems: http://perl.apache.org/bugs/
                            Mail list info: http://perl.apache.org/maillist/modperl.html
                            List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                          • Glenn Strauss
                            On Mon, Aug 02, 2004 at 11:55:56AM -0700, Stas Bekman wrote: [...] ... I mentioned to Geoff off-list about possibly using a flush bucket, since I don t have a
                            Message 13 of 25 , Aug 3, 2004
                            • 0 Attachment
                              On Mon, Aug 02, 2004 at 11:55:56AM -0700, Stas Bekman wrote:
                              [...]
                              > Actually I the problem I saw was exactly what Boris was talking about:
                              > The C-L header wasn't there. The test simply exercises 4 different
                              > combinations of sending and not sending C-L header and content.
                              >
                              > >my point was that as long as apache does whatever it does
                              > >consistently between GET and HEAD it's not a bug. but maybe I'm just not
                              > >seeing it, so if you guys grok all of that and still see an issue, let's
                              > >flesh it out.
                              >
                              > I think it's there. The test that shows that is:
                              >
                              > {
                              > # if the response handler sends no data, and sets C-L header,
                              > # the client doesn't get C-L header
                              > my $uri = "$location?set_content_length";
                              > my $res = $method->($uri);
                              > ok t_cmp $res->code, 200, "$method $uri code";
                              > ok t_cmp $res->header('Content-Length'), undef, "$method $uri
                              > C-L header";
                              > ok t_cmp $res->content, "", "$method $uri content";
                              > }
                              >
                              > As I didn't know what's the right thing that should happen, I didn't
                              > make it a failure. But it probabaly shouldn't be undef in the C-L check.
                              >
                              > But the fact that sending a bogus char and a different C-L header works
                              > with HEAD (gets the C-L header through), whereas not sending any output
                              > breaks, seems like a bug to me. As mentioned earlier I think the bug is
                              > in the headers_out filter, which received EOS (since on HEAD apache
                              > scratches the response body) and no body. So it takes the liberty to
                              > nuke the C-L header, which I'm not sure is a good thing. When we send
                              > some body, headers_out sends the headers before seeing EOS and therefore
                              > it can't tell whether the body is coming or not, so it leaves the C-L
                              > header alone. Which is at least inconsistent.


                              I mentioned to Geoff off-list about possibly using a flush bucket,
                              since I don't have a test setup ready (just replaced my dead
                              laptop hard drive)

                              I wrote:
                              > > I used to make good use of $r->send_http_header() in my MP1 handlers.
                              > > This method isn't available in MP2 (why?), though I could picture it
                              > > replaced by sending a flush bucket down the filter chain.
                              >
                              On Tue, Aug 03, 2004 at 10:04:38AM -0400, Geoffrey Young wrote:
                              > apache now sends the headers for you via the header filter. if you were
                              > allowed to send your own headers you might send them before a filter ran
                              > that would alter the headers. for instance, mod_include removes the ETag
                              > header.

                              Would a flush bucket be a great way to say "send headers now"?
                              So if you're not going to send the actual content down the filter chain
                              on a HEAD request, would sending a flush bucket make things happy?

                              IIF this works, can I make a request the $r->send_http_header() be
                              implemented in Apache2 to do just that? (create and pass a flush bucket)

                              Cheers,
                              Glenn

                              --
                              Report problems: http://perl.apache.org/bugs/
                              Mail list info: http://perl.apache.org/maillist/modperl.html
                              List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                            • Stas Bekman
                              ... Thanks Glenn, calling $r- rflush works perfectly fine to cause user-preset C-L header to be delivered to the client on HEAD request. ... I think this is
                              Message 14 of 25 , Aug 4, 2004
                              • 0 Attachment
                                Glenn Strauss wrote:
                                > On Mon, Aug 02, 2004 at 11:55:56AM -0700, Stas Bekman wrote:
                                > [...]

                                > I mentioned to Geoff off-list about possibly using a flush bucket,
                                > since I don't have a test setup ready (just replaced my dead
                                > laptop hard drive)

                                Thanks Glenn, calling $r->rflush works perfectly fine to cause
                                user-preset C-L header to be delivered to the client on HEAD request.

                                > I wrote:
                                >
                                >>>I used to make good use of $r->send_http_header() in my MP1 handlers.
                                >>>This method isn't available in MP2 (why?), though I could picture it
                                >>>replaced by sending a flush bucket down the filter chain.
                                >>
                                > On Tue, Aug 03, 2004 at 10:04:38AM -0400, Geoffrey Young wrote:
                                >
                                >>apache now sends the headers for you via the header filter. if you were
                                >>allowed to send your own headers you might send them before a filter ran
                                >>that would alter the headers. for instance, mod_include removes the ETag
                                >>header.

                                I think this is not the reason why ap_send_http_header was removed from
                                Apache2 API. It's simply not needed because the headers_out filter will
                                inject the headers as soon as some body is sent. Your example of
                                mod_include doesn't work if the response handler does:

                                $r->print("foo bar");
                                $r->rflush; # or if the previous print was big enough to cause flush
                                $r->print(q[<!--#include virtual="/foo" -->]);

                                > Would a flush bucket be a great way to say "send headers now"?
                                > So if you're not going to send the actual content down the filter chain
                                > on a HEAD request, would sending a flush bucket make things happy?

                                It does.

                                > IIF this works, can I make a request the $r->send_http_header() be
                                > implemented in Apache2 to do just that? (create and pass a flush bucket)

                                Absolutely. But it'd be nice if you could first ask the httpd-dev list:
                                http://httpd.apache.org/lists.html#http-dev
                                if they are interested in adding the C API to do that.

                                If httpd-dev doesn't do it, I suppose we can still implement it in mp2.

                                --
                                __________________________________________________________________
                                Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                                http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                                mailto:stas@... http://use.perl.org http://apacheweek.com
                                http://modperlbook.org http://apache.org http://ticketmaster.com

                                --
                                Report problems: http://perl.apache.org/bugs/
                                Mail list info: http://perl.apache.org/maillist/modperl.html
                                List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                              • Stas Bekman
                                Stas Bekman wrote: [...] ... On a second thought, I think it s better not to add this special method unless Apache does that. Since in 99.99% cases one should
                                Message 15 of 25 , Aug 4, 2004
                                • 0 Attachment
                                  Stas Bekman wrote:
                                  [...]
                                  >> Would a flush bucket be a great way to say "send headers now"?
                                  >> So if you're not going to send the actual content down the filter chain
                                  >> on a HEAD request, would sending a flush bucket make things happy?
                                  >
                                  >
                                  > It does.
                                  >
                                  >> IIF this works, can I make a request the $r->send_http_header() be
                                  >> implemented in Apache2 to do just that? (create and pass a flush bucket)
                                  >
                                  >
                                  > Absolutely. But it'd be nice if you could first ask the httpd-dev list:
                                  > http://httpd.apache.org/lists.html#http-dev
                                  > if they are interested in adding the C API to do that.
                                  >
                                  > If httpd-dev doesn't do it, I suppose we can still implement it in mp2.

                                  On a second thought, I think it's better not to add this special method
                                  unless Apache does that. Since in 99.99% cases one should not cause an
                                  early headers sending, and for those special cases which go against the
                                  Apache spec (sending no body for HEAD requests) we will simply document
                                  a workaround. This is because the workaround may stop working in the
                                  future and we will find ourselves in a bad situation.

                                  If Apache adds this method and documents it usage for the purposes
                                  similar to one discussed in this thread, then Apache will need to ensure
                                  that it'll work in the future. Therefore if you think send_http_header()
                                  is needed, please discuss that on the quoted above list, and depending
                                  on the outcome we will either provide the perl interface to it or simply
                                  document a workaround, which may or may not stop working in the future.

                                  --
                                  __________________________________________________________________
                                  Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                                  http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                                  mailto:stas@... http://use.perl.org http://apacheweek.com
                                  http://modperlbook.org http://apache.org http://ticketmaster.com

                                  --
                                  Report problems: http://perl.apache.org/bugs/
                                  Mail list info: http://perl.apache.org/maillist/modperl.html
                                  List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                • Stas Bekman
                                  ... Thanks Geoff for looking at the source code and giving us the full story. I ve now converted it into the comments in the t/apache/content_length_header.t
                                  Message 16 of 25 , Aug 4, 2004
                                  • 0 Attachment
                                    Geoffrey Young wrote:
                                    >
                                    >>As mentioned earlier I think the bug is
                                    >>in the headers_out filter, which received EOS (since on HEAD apache
                                    >>scratches the response body) and no body. So it takes the liberty to
                                    >>nuke the C-L header, which I'm not sure is a good thing. When we send
                                    >>some body, headers_out sends the headers before seeing EOS and therefore
                                    >>it can't tell whether the body is coming or not, so it leaves the C-L
                                    >>header alone. Which is at least inconsistent.
                                    >
                                    >
                                    > ok, I think I see what is going on, but it's different than what you see :)
                                    >
                                    > first, apache calls the C-L filter, which has no regard for any existing C-L
                                    > header and calculates its own if it can. so, in your test case the call to
                                    > $r->set_content_length(25) is overwritten by the real C-L of zero, as
                                    > calculated by the C-L filter.
                                    >
                                    > next the header filter kicks in. inside the header filter is this bit of logic:
                                    >
                                    > if (r->header_only
                                    > && (clheader = apr_table_get(r->headers_out, "Content-Length"))
                                    > && !strcmp(clheader, "0")) {
                                    > apr_table_unset(r->headers_out, "Content-Length");
                                    > }
                                    >
                                    > which treats HEAD requests with a C-L of zero as a special case. and, in
                                    > fact, that is what you have sent it, at least in the test case. this is why
                                    > GET and HEAD differ in this instance, because you haven't sent any real
                                    > content and a C-L of 0 is special.
                                    >
                                    > so, I don't see the bug. well, I do see it, except that it is intentionally
                                    > coded to be as such, which turns the bug into a feature.

                                    Thanks Geoff for looking at the source code and giving us the full
                                    story. I've now converted it into the comments in the
                                    t/apache/content_length_header.t test and I also used Glenn's suggestion
                                    to use $r->rflush, which gives us:

                                    # 2. even though the spec says that content handlers should send an
                                    # identical response for GET and HEAD requests, some folks try to
                                    # avoid the overhead of generating the response body, which Apache is
                                    # going to discard anyway for HEAD requests. The following discussion
                                    # assumes that we deal with a HEAD request.
                                    #
                                    # When Apache sees EOS and no headers and no response body were sent,
                                    # ap_content_length_filter (httpd-2.0/server/protocol.c) sets C-L to
                                    # 0. Later on ap_http_header_filter
                                    # (httpd-2.0/modules/http/http_protocol.c) removes the C-L header for
                                    # the HEAD requests
                                    #
                                    # the workaround is to force the sending of the response headers,
                                    # before EOS was sent. The simplest solution is to use rflush():
                                    #
                                    # if ($r->header_only) { # HEAD
                                    # $body_len = calculate_body_len();
                                    # $r->set_content_length($body_len);
                                    # $r->rflush;
                                    # }
                                    # else { # GET
                                    # # generate and send the body
                                    # }
                                    #
                                    # now if the handler sets the C-L header it'll be delivered to the
                                    # client unmodified.


                                    --
                                    __________________________________________________________________
                                    Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                                    http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                                    mailto:stas@... http://use.perl.org http://apacheweek.com
                                    http://modperlbook.org http://apache.org http://ticketmaster.com

                                    --
                                    Report problems: http://perl.apache.org/bugs/
                                    Mail list info: http://perl.apache.org/maillist/modperl.html
                                    List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                  • Geoffrey Young
                                    ... I fully agree. ... right - apache expects you to not do anything special for HEAD requests anymore, so if you mess with that contract you are messing with
                                    Message 17 of 25 , Aug 4, 2004
                                    • 0 Attachment
                                      >> If httpd-dev doesn't do it, I suppose we can still implement it in mp2.
                                      >
                                      >
                                      > On a second thought, I think it's better not to add this special method
                                      > unless Apache does that.

                                      I fully agree.

                                      > Since in 99.99% cases one should not cause an
                                      > early headers sending, and for those special cases which go against the
                                      > Apache spec (sending no body for HEAD requests) we will simply document
                                      > a workaround. This is because the workaround may stop working in the
                                      > future and we will find ourselves in a bad situation.

                                      right - apache expects you to not do anything special for HEAD requests
                                      anymore, so if you mess with that contract you are messing with an
                                      unsupported feature, and mod_perl has no business supporting unsupported
                                      apache features through a new, mod_perl-specific API.

                                      --Geoff

                                      --
                                      Report problems: http://perl.apache.org/bugs/
                                      Mail list info: http://perl.apache.org/maillist/modperl.html
                                      List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                    • Stas Bekman
                                      This thread has been documented at: http://perl.apache.org/docs/2.0/user/handlers/http.html#C_Content_Length__Response_Header --
                                      Message 18 of 25 , Aug 8, 2004
                                      • 0 Attachment
                                        This thread has been documented at:
                                        http://perl.apache.org/docs/2.0/user/handlers/http.html#C_Content_Length__Response_Header

                                        --
                                        __________________________________________________________________
                                        Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                                        http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                                        mailto:stas@... http://use.perl.org http://apacheweek.com
                                        http://modperlbook.org http://apache.org http://ticketmaster.com

                                        --
                                        Report problems: http://perl.apache.org/bugs/
                                        Mail list info: http://perl.apache.org/maillist/modperl.html
                                        List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                      Your message has been successfully submitted and would be delivered to recipients shortly.