Loading ...
Sorry, an error occurred while loading the content.

MPM worker and ithreads

Expand Messages
  • Stefan Cars
    Hi! Anyone that has any good links or any good input on using the worker MPM and ithreads with mod_perl 2 and Apache 2 Kind Regards, Stefan Cars -- Report
    Message 1 of 22 , Feb 20, 2004
    • 0 Attachment
      Hi!

      Anyone that has any good links or any good input on using the worker MPM
      and ithreads with mod_perl 2 and Apache 2

      Kind Regards,
      Stefan Cars

      --
      Report problems: http://perl.apache.org/bugs/
      Mail list info: http://perl.apache.org/maillist/modperl.html
      List etiquette: http://perl.apache.org/maillist/email-etiquette.html
    • Stas Bekman
      ... You could be a bit more specific, Stafan. What kind of problems do you have? You also have to tell us more about your environment, which is done the best
      Message 2 of 22 , Feb 20, 2004
      • 0 Attachment
        Stefan Cars wrote:
        > Hi!
        >
        > Anyone that has any good links or any good input on using the worker MPM
        > and ithreads with mod_perl 2 and Apache 2

        You could be a bit more specific, Stafan. What kind of problems do you have?
        You also have to tell us more about your environment, which is done the best
        by calling mp2bug or t/REPORT
        http://perl.apache.org/docs/2.0/user/help/help.html#Important_Information

        __________________________________________________________________
        Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
        http://stason.org/ mod_perl Guide ---> http://perl.apache.org
        mailto:stas@... http://use.perl.org http://apacheweek.com
        http://modperlbook.org http://apache.org http://ticketmaster.com

        --
        Report problems: http://perl.apache.org/bugs/
        Mail list info: http://perl.apache.org/maillist/modperl.html
        List etiquette: http://perl.apache.org/maillist/email-etiquette.html
      • Stefan Cars
        Hi! I don t have any problems, I m just interested in what people think of the performance vs. preforking and if there is any problems using worker and
        Message 3 of 22 , Feb 21, 2004
        • 0 Attachment
          Hi!

          I don't have any problems, I'm just interested in what people think of
          the performance vs. preforking and if there is any problems using worker
          and ithreads ?

          On Fri, 20 Feb 2004, Stas Bekman wrote:

          > Stefan Cars wrote:
          > > Hi!
          > >
          > > Anyone that has any good links or any good input on using the worker MPM
          > > and ithreads with mod_perl 2 and Apache 2
          >
          > You could be a bit more specific, Stafan. What kind of problems do you have?
          > You also have to tell us more about your environment, which is done the best
          > by calling mp2bug or t/REPORT
          > http://perl.apache.org/docs/2.0/user/help/help.html#Important_Information
          >
          > __________________________________________________________________
          > Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
          > http://stason.org/ mod_perl Guide ---> http://perl.apache.org
          > mailto:stas@... http://use.perl.org http://apacheweek.com
          > http://modperlbook.org http://apache.org http://ticketmaster.com
          >

          --
          Stefan Cars
          Snowfall Communications
          Tel: +46 (0)18 430 80 50 - Direct: +46 (0)18 430 80 51
          Mobile: +46 (0)708 44 36 00 - Fax: +46 (0)708 44 36 04



          ______________________________________________________________________
          SNOWFALL DISCLAIMER:
          The information contained in this email and in any
          attachments is confidential and may be privileged. If you are not the
          intended recipient, please destroy this message and notify the sender
          immediately. You should not retain, copy or use this email for any
          purpose, nor disclose all or any part of its content to any other person.
          Any views expressed in this message are those of the individual sender,
          except where the sender specifically states them to be the views of
          Snowfall Communications.

          Snowfall Communications monitors the content of emails sent and received
          via its network for unauthorised use and for other lawful business
          purposes. The contents of an attachment to this email may contain viruses
          which could damage your computer system. While Snowfall Communications has
          taken every reasonable precaution to minimise this risk, we cannot accept
          liability for any damage which you sustain as a result of software
          viruses. You should carry out your own virus checks before opening the
          attachment.

          --
          Report problems: http://perl.apache.org/bugs/
          Mail list info: http://perl.apache.org/maillist/modperl.html
          List etiquette: http://perl.apache.org/maillist/email-etiquette.html
        • Richard F. Rebel
          Hello, I am using ap2 + mp2 with the worker mpm and ithreads to serve about 12 million dynamically generated pages per day. This includes RPC calls via HTTP
          Message 4 of 22 , Feb 23, 2004
          • 0 Attachment
            Hello,

            I am using ap2 + mp2 with the worker mpm and ithreads to serve about 12
            million dynamically generated pages per day. This includes RPC calls
            via HTTP and database queries. I am currently experimenting with
            creating pools of threads inside the perl interpreter for handling
            communicating with multiple remote machines simultaniously.

            My project would not be possible using prefork without a huge increase
            in hardware expense (the reason we tried using worker mpm).

            Performance is good on decent hardware, sub second page generation
            times. I have noticed that for my app, prefork was a smidgen faster.

            That said the following should be said as well:

            Make sure (atleast on Linux) that you have set your number of available
            threads high enough that requests do not have to wait for a new thread
            to be created. Your performance will suck otherwise as new thread
            creation is slower than one would expect. This is true for apache2
            without mod_perl as well btw but not quite as bad..

            Apache2 still has bugs, so does mp2, be careful. I have identified at
            at least one run away memory leak that I can't track down and apache2
            segfaults on occasion. Isolating such things in busy threaded apps is
            *very* difficult.

            ithreads in perl are very slow to start and the implementation is a bit
            buggy with spotty documentation. Eg, you *can NOT* return a list from a
            thread, you may return exactly one scalar ref to whatever, the docs are
            *wrong* and the snots on P5P have yet to even admit (atleast to me, I
            stopped following the issue a while ago, may be fixed now) that this is
            even a bug or a problem.

            Best,

            Richard F. Rebel



            On Sat, 2004-02-21 at 06:07, Stefan Cars wrote:
            > Hi!
            >
            > I don't have any problems, I'm just interested in what people think of
            > the performance vs. preforking and if there is any problems using worker
            > and ithreads ?
            >
            > On Fri, 20 Feb 2004, Stas Bekman wrote:
            >
            > > Stefan Cars wrote:
            > > > Hi!
            > > >
            > > > Anyone that has any good links or any good input on using the worker MPM
            > > > and ithreads with mod_perl 2 and Apache 2
            > >
            > > You could be a bit more specific, Stafan. What kind of problems do you have?
            > > You also have to tell us more about your environment, which is done the best
            > > by calling mp2bug or t/REPORT
            > > http://perl.apache.org/docs/2.0/user/help/help.html#Important_Information
            > >
            > > __________________________________________________________________
            > > Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
            > > http://stason.org/ mod_perl Guide ---> http://perl.apache.org
            > > mailto:stas@... http://use.perl.org http://apacheweek.com
            > > http://modperlbook.org http://apache.org http://ticketmaster.com
            > >
            >
            > --
            > Stefan Cars
            > Snowfall Communications
            > Tel: +46 (0)18 430 80 50 - Direct: +46 (0)18 430 80 51
            > Mobile: +46 (0)708 44 36 00 - Fax: +46 (0)708 44 36 04
            >
            >
            >
            > ______________________________________________________________________
            > SNOWFALL DISCLAIMER:
            > The information contained in this email and in any
            > attachments is confidential and may be privileged. If you are not the
            > intended recipient, please destroy this message and notify the sender
            > immediately. You should not retain, copy or use this email for any
            > purpose, nor disclose all or any part of its content to any other person.
            > Any views expressed in this message are those of the individual sender,
            > except where the sender specifically states them to be the views of
            > Snowfall Communications.
            >
            > Snowfall Communications monitors the content of emails sent and received
            > via its network for unauthorised use and for other lawful business
            > purposes. The contents of an attachment to this email may contain viruses
            > which could damage your computer system. While Snowfall Communications has
            > taken every reasonable precaution to minimise this risk, we cannot accept
            > liability for any damage which you sustain as a result of software
            > viruses. You should carry out your own virus checks before opening the
            > attachment.
            --
            Richard F. Rebel
            rrebel@...
            t. 212.239.0000
          • Ged Haywood
            Hi there, ... Thanks very much for that report, it s really useful. I see your experience with the -er- people at P5P was similar to mine. 73, Ged. -- Report
            Message 5 of 22 , Feb 23, 2004
            • 0 Attachment
              Hi there,

              On Mon, 23 Feb 2004, Richard F. Rebel wrote:

              > I am using ap2 + mp2 with the worker mpm and ithreads to serve about 12
              > million dynamically generated pages per day. ... expense ... performance
              > ... bugs ... ithreads ... snots on P5P

              Thanks very much for that report, it's really useful.

              I see your experience with the -er- people at P5P was similar to mine.

              73,
              Ged.

              --
              Report problems: http://perl.apache.org/bugs/
              Mail list info: http://perl.apache.org/maillist/modperl.html
              List etiquette: http://perl.apache.org/maillist/email-etiquette.html
            • Stas Bekman
              ... That s very untrue, guys. p5p cares a lot for mod_perl and helps a lot to fix things so that mod_perl will be happy. Not once a major release was delayed
              Message 6 of 22 , Feb 23, 2004
              • 0 Attachment
                Ged Haywood wrote:
                > Hi there,
                >
                > On Mon, 23 Feb 2004, Richard F. Rebel wrote:
                >
                >
                >>I am using ap2 + mp2 with the worker mpm and ithreads to serve about 12
                >>million dynamically generated pages per day. ... expense ... performance
                >>... bugs ... ithreads ... snots on P5P
                >
                >
                > Thanks very much for that report, it's really useful.
                >
                > I see your experience with the -er- people at P5P was similar to mine.

                That's very untrue, guys. p5p cares a lot for mod_perl and helps a lot to fix
                things so that mod_perl will be happy. Not once a major release was delayed
                because of problems with mod_perl. I wish httpd-dev cared about mod_perl just
                a tiny fraction of what p5p does.

                The issue with ithreads support/maintenance is different: At the moment the
                ithreads package has no maintainer, so any fixes that you see at all are
                spotty. Arthur Bergman has left to work on Ponie (perl5 on parrot
                http://www.poniecode.org/) and noone has stepped up to replace him.
                So if someone feels like picking up an important sub-project, by all means do
                so. It requires understanding of threads, XS and Perl guts.

                __________________________________________________________________
                Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                mailto:stas@... http://use.perl.org http://apacheweek.com
                http://modperlbook.org http://apache.org http://ticketmaster.com

                --
                Report problems: http://perl.apache.org/bugs/
                Mail list info: http://perl.apache.org/maillist/modperl.html
                List etiquette: http://perl.apache.org/maillist/email-etiquette.html
              • Stas Bekman
                ... Thanks Richard. We are always looking for success stories, if you wish to contribute one. Please see: http://perl.apache.org/outstanding/index.html
                Message 7 of 22 , Feb 23, 2004
                • 0 Attachment
                  Richard F. Rebel wrote:
                  > Hello,
                  >
                  > I am using ap2 + mp2 with the worker mpm and ithreads to serve about 12
                  > million dynamically generated pages per day. This includes RPC calls
                  > via HTTP and database queries. I am currently experimenting with
                  > creating pools of threads inside the perl interpreter for handling
                  > communicating with multiple remote machines simultaniously.
                  >
                  > My project would not be possible using prefork without a huge increase
                  > in hardware expense (the reason we tried using worker mpm).
                  >
                  > Performance is good on decent hardware, sub second page generation
                  > times. I have noticed that for my app, prefork was a smidgen faster.

                  Thanks Richard. We are always looking for success stories, if you wish to
                  contribute one. Please see:
                  http://perl.apache.org/outstanding/index.html

                  __________________________________________________________________
                  Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                  http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                  mailto:stas@... http://use.perl.org http://apacheweek.com
                  http://modperlbook.org http://apache.org http://ticketmaster.com

                  --
                  Report problems: http://perl.apache.org/bugs/
                  Mail list info: http://perl.apache.org/maillist/modperl.html
                  List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                • Richard F. Rebel
                  ... There is a general attitude to posters on that list, we all know it. I don t think anyone meant to say they didn t care about mp but rather that owning up
                  Message 8 of 22 , Feb 23, 2004
                  • 0 Attachment
                    On Mon, 2004-02-23 at 14:05, Stas Bekman wrote:
                    > Ged Haywood wrote:
                    > >
                    > > I see your experience with the -er- people at P5P was similar to mine.
                    >
                    > That's very untrue, guys. p5p cares a lot for mod_perl and helps a lot to fix
                    > things so that mod_perl will be happy. Not once a major release was delayed
                    > because of problems with mod_perl. I wish httpd-dev cared about mod_perl just
                    > a tiny fraction of what p5p does.

                    There is a general attitude to posters on that list, we all know it. I
                    don't think anyone meant to say they didn't care about mp but rather
                    that owning up to problems in perl itself is often low on the priority
                    list. Eg, broken select().

                    > The issue with ithreads support/maintenance is different: At the moment the
                    > ithreads package has no maintainer, so any fixes that you see at all are
                    > spotty. Arthur Bergman has left to work on Ponie (perl5 on parrot
                    > http://www.poniecode.org/) and noone has stepped up to replace him.

                    No one mentioned this during my posts.

                    > So if someone feels like picking up an important sub-project, by all means do
                    > so. It requires understanding of threads, XS and Perl guts.

                    Well, that would be the problem... XS + Perl Guts. Talk about arcane.

                    --
                    Richard F. Rebel
                    rrebel@...
                    t. 212.239.0000
                  • Perrin Harkins
                    ... What s the reason for this? Are you saving memory by using the worker MPM? I thought the consensus was that perl threads actually use more memory than
                    Message 9 of 22 , Feb 23, 2004
                    • 0 Attachment
                      On Mon, 2004-02-23 at 10:03, Richard F. Rebel wrote:
                      > I am using ap2 + mp2 with the worker mpm and ithreads to serve about 12
                      > million dynamically generated pages per day. This includes RPC calls
                      > via HTTP and database queries. I am currently experimenting with
                      > creating pools of threads inside the perl interpreter for handling
                      > communicating with multiple remote machines simultaniously.
                      >
                      > My project would not be possible using prefork without a huge increase
                      > in hardware expense (the reason we tried using worker mpm).

                      What's the reason for this? Are you saving memory by using the worker
                      MPM? I thought the consensus was that perl threads actually use more
                      memory than pre-fork at this point.

                      - Perrin


                      --
                      Report problems: http://perl.apache.org/bugs/
                      Mail list info: http://perl.apache.org/maillist/modperl.html
                      List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                    • Richard F. Rebel
                      ... Hello Perrin, Umm, maybe we are talking about different things. If I run my application with 500+ httpd s in the process list using prefork, it uses a lot
                      Message 10 of 22 , Feb 23, 2004
                      • 0 Attachment
                        On Mon, 2004-02-23 at 16:04, Perrin Harkins wrote:
                        > On Mon, 2004-02-23 at 10:03, Richard F. Rebel wrote:
                        > > I am using ap2 + mp2 with the worker mpm and ithreads to serve about 12
                        > > million dynamically generated pages per day. This includes RPC calls
                        > > via HTTP and database queries. I am currently experimenting with
                        > > creating pools of threads inside the perl interpreter for handling
                        > > communicating with multiple remote machines simultaniously.
                        > >
                        > > My project would not be possible using prefork without a huge increase
                        > > in hardware expense (the reason we tried using worker mpm).
                        >
                        > What's the reason for this? Are you saving memory by using the worker
                        > MPM? I thought the consensus was that perl threads actually use more
                        > memory than pre-fork at this point.

                        Hello Perrin,

                        Umm, maybe we are talking about different things. If I run my
                        application with 500+ httpd's in the process list using prefork, it uses
                        a lot more memory than running 10 httpds with 64 threads each using
                        worker. It also gets worse over time (as shared pages get altered).

                        Did I miss something or did I do something wrong?

                        > - Perrin
                        --
                        Richard F. Rebel
                        rrebel@...
                        t. 212.239.0000
                      • Perrin Harkins
                        ... I m not sure you did anything wrong, but your results are very different from what other people have reported. In general, because perl threads don t
                        Message 11 of 22 , Feb 23, 2004
                        • 0 Attachment
                          On Mon, 2004-02-23 at 16:40, Richard F. Rebel wrote:
                          > Umm, maybe we are talking about different things. If I run my
                          > application with 500+ httpd's in the process list using prefork, it uses
                          > a lot more memory than running 10 httpds with 64 threads each using
                          > worker. It also gets worse over time (as shared pages get altered).
                          >
                          > Did I miss something or did I do something wrong?

                          I'm not sure you did anything wrong, but your results are very different
                          from what other people have reported. In general, because perl threads
                          don't actually share anything (except opcode trees) unless you tell them
                          to, all the data gets copied to each thread, which ends up taking up
                          more memory than the prefork model with COW.

                          Maybe you have the PerlInterp* settings set low so that you actually
                          have fewer interpreters running than you did when you were running
                          prefork? If so, that's similar to using prefork with a reverse proxy in
                          front.

                          - Perrin


                          --
                          Report problems: http://perl.apache.org/bugs/
                          Mail list info: http://perl.apache.org/maillist/modperl.html
                          List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                        • Nigel Hamilton
                          ... Hi Richard, If you don t mind me asking how much RAM do you need to run 500 preforked httpds? Nige -- Nigel Hamilton Turbo10 Metasearch Engine email:
                          Message 12 of 22 , Feb 23, 2004
                          • 0 Attachment
                            >
                            > Umm, maybe we are talking about different things. If I run my
                            > application with 500+ httpd's in the process list using prefork, it uses
                            > a lot more memory than running 10 httpds with 64 threads each using
                            > worker. It also gets worse over time (as shared pages get altered).
                            >

                            Hi Richard,

                            If you don't mind me asking how much RAM do you need to run 500
                            preforked httpds?


                            Nige



                            --
                            Nigel Hamilton
                            Turbo10 Metasearch Engine

                            email: nigel@...
                            tel: +44 (0) 207 987 5460
                            fax: +44 (0) 207 987 5468
                            ________________________________________________________________________________
                            http://turbo10.com Search Deeper. Browse Faster.


                            --
                            Report problems: http://perl.apache.org/bugs/
                            Mail list info: http://perl.apache.org/maillist/modperl.html
                            List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                          • Richard F. Rebel
                            ... Interesting. I just noticed documentation on the web site about these directives. Were these docs here about 1 year ago (when I wrote this app???). ... I
                            Message 13 of 22 , Feb 23, 2004
                            • 0 Attachment
                              On Mon, 2004-02-23 at 16:49, Perrin Harkins wrote:
                              > On Mon, 2004-02-23 at 16:40, Richard F. Rebel wrote:
                              > > Umm, maybe we are talking about different things. If I run my
                              > > application with 500+ httpd's in the process list using prefork, it uses
                              > > a lot more memory than running 10 httpds with 64 threads each using
                              > > worker. It also gets worse over time (as shared pages get altered).
                              > >
                              > > Did I miss something or did I do something wrong?
                              >
                              > I'm not sure you did anything wrong, but your results are very different
                              > from what other people have reported. In general, because perl threads
                              > don't actually share anything (except opcode trees) unless you tell them
                              > to, all the data gets copied to each thread, which ends up taking up
                              > more memory than the prefork model with COW.

                              Interesting. I just noticed documentation on the web site about these
                              directives. Were these docs here about 1 year ago (when I wrote this
                              app???).

                              > Maybe you have the PerlInterp* settings set low so that you actually
                              > have fewer interpreters running than you did when you were running
                              > prefork? If so, that's similar to using prefork with a reverse proxy in
                              > front.

                              I have *no* perlInterp* configuration directives.

                              > - Perrin
                              --
                              Richard F. Rebel
                              rrebel@...
                              t. 212.239.0000
                            • Perrin Harkins
                              ... Those have been there for years. I remember that stuff being in Doug s slides before the first alpha release of mp2. ... Judging by the defaults, it
                              Message 14 of 22 , Feb 23, 2004
                              • 0 Attachment
                                On Mon, 2004-02-23 at 17:16, Richard F. Rebel wrote:
                                > Interesting. I just noticed documentation on the web site about these
                                > directives. Were these docs here about 1 year ago (when I wrote this
                                > app???).

                                Those have been there for years. I remember that stuff being in Doug's
                                slides before the first alpha release of mp2.

                                > I have *no* perlInterp* configuration directives.

                                Judging by the defaults, it sounds like you have a total of 50 perl
                                interpreters then, and all other threads are either serving static files
                                or queuing in line for use of an interpreter.

                                You would get basically the same effect by running prefork with
                                MaxClients set to 50, and a reverse proxy in front. Actually, you could
                                probably run a lot more interpreters with prefork, because it is
                                currently MORE memory efficient than threads. I guess you gain a bit of
                                performance by not going through the proxy, but lose some scalability by
                                not having the proxy buffer your output to slow clients.

                                - Perrin


                                --
                                Report problems: http://perl.apache.org/bugs/
                                Mail list info: http://perl.apache.org/maillist/modperl.html
                                List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                              • Richard F. Rebel
                                ... I feel like king dork. Eh well, do I get points for making a prod system run with mp2 and mpm-worker? ... This explains some of the sporadic slow results
                                Message 15 of 22 , Feb 23, 2004
                                • 0 Attachment
                                  On Mon, 2004-02-23 at 18:01, Perrin Harkins wrote:
                                  > On Mon, 2004-02-23 at 17:16, Richard F. Rebel wrote:
                                  > > Interesting. I just noticed documentation on the web site about these
                                  > > directives. Were these docs here about 1 year ago (when I wrote this
                                  > > app???).
                                  >
                                  > Those have been there for years. I remember that stuff being in Doug's
                                  > slides before the first alpha release of mp2.

                                  I feel like king dork. Eh well, do I get points for making a prod
                                  system run with mp2 and mpm-worker?

                                  > > I have *no* perlInterp* configuration directives.
                                  >
                                  > Judging by the defaults, it sounds like you have a total of 50 perl
                                  > interpreters then, and all other threads are either serving static files
                                  > or queuing in line for use of an interpreter.

                                  This explains some of the sporadic slow results I get on occasion that
                                  don't seem to be related to anything external. Apache was having to on
                                  occasion start new interpreter threads.

                                  > You would get basically the same effect by running prefork with
                                  > MaxClients set to 50, and a reverse proxy in front. Actually, you could
                                  > probably run a lot more interpreters with prefork, because it is
                                  > currently MORE memory efficient than threads. I guess you gain a bit of
                                  > performance by not going through the proxy, but lose some scalability by
                                  > not having the proxy buffer your output to slow clients.

                                  Most of our clients are *slow*, so perhaps this is why things seem to
                                  work so well. I have increased the number of interpreters and have
                                  noticed a decrease in cpu usage and an improvement in performance in
                                  general. Thanks.

                                  Now, does this sort of explain why I was seeing more memory usage with
                                  prefork, slow clients? When exactly is the perl interpreter put back
                                  into the 'free' list? Does apache spoon feed the slow clients freeing
                                  up the interpreter for other requests after you return from your
                                  handler?

                                  > - Perrin
                                  --
                                  Richard F. Rebel <rrebel@...>
                                  WhenU.com
                                • Perrin Harkins
                                  ... Certainly. We are all eager for this kind of info. ... Actually, if your clients are slow you would be better off with a reverse proxy to do buffering.
                                  Message 16 of 22 , Feb 23, 2004
                                  • 0 Attachment
                                    On Mon, 2004-02-23 at 19:37, Richard F. Rebel wrote:
                                    > Eh well, do I get points for making a prod
                                    > system run with mp2 and mpm-worker?

                                    Certainly. We are all eager for this kind of info.

                                    > Most of our clients are *slow*, so perhaps this is why things seem to
                                    > work so well.

                                    Actually, if your clients are slow you would be better off with a
                                    reverse proxy to do buffering.

                                    > I have increased the number of interpreters and have
                                    > noticed a decrease in cpu usage and an improvement in performance in
                                    > general.

                                    Cool.

                                    > Now, does this sort of explain why I was seeing more memory usage with
                                    > prefork, slow clients?

                                    That was because you were running 500 interpreters instead of 50.

                                    > When exactly is the perl interpreter put back
                                    > into the 'free' list?

                                    More of a question for someone closer to the code than me, but I thought
                                    that the interpreter was tied up until the server finishes the request,
                                    meaning that slow clients will keep an interpreter tied up for a long
                                    time.

                                    - Perrin


                                    --
                                    Report problems: http://perl.apache.org/bugs/
                                    Mail list info: http://perl.apache.org/maillist/modperl.html
                                    List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                  • Richard F. Rebel
                                    ... Yay, points. ... This makes sense but if the perl interpreters are tied up as you mentioned below, I d need a separate proxy. We do this on our general
                                    Message 17 of 22 , Feb 23, 2004
                                    • 0 Attachment
                                      On Mon, 2004-02-23 at 19:48, Perrin Harkins wrote:
                                      > On Mon, 2004-02-23 at 19:37, Richard F. Rebel wrote:
                                      > > Eh well, do I get points for making a prod
                                      > > system run with mp2 and mpm-worker?
                                      >
                                      > Certainly. We are all eager for this kind of info.

                                      Yay, points.

                                      > > Most of our clients are *slow*, so perhaps this is why things seem to
                                      > > work so well.
                                      >
                                      > Actually, if your clients are slow you would be better off with a
                                      > reverse proxy to do buffering.

                                      This makes sense but if the perl interpreters are tied up as you
                                      mentioned below, I'd need a separate proxy. We do this on our general
                                      purpose app servers already so I can do some experimentation and report
                                      back.

                                      > > Now, does this sort of explain why I was seeing more memory usage with
                                      > > prefork, slow clients?
                                      >
                                      > That was because you were running 500 interpreters instead of 50.

                                      Speaking of, does anyone know of a way to tell exactly how many
                                      interpreters are running in a given process?

                                      Again, Thanks!

                                      > > When exactly is the perl interpreter put back
                                      > > into the 'free' list?
                                      >
                                      > More of a question for someone closer to the code than me, but I thought
                                      > that the interpreter was tied up until the server finishes the request,
                                      > meaning that slow clients will keep an interpreter tied up for a long
                                      > time.
                                      >
                                      > - Perrin
                                      --
                                      Richard F. Rebel <rrebel@...>
                                      WhenU.com
                                    • modperl@att.net
                                      Somehow a high volume site for your reference: System: P4 1.6G, 1G memory, Linux 7.2, Perl 5.6, customized kernel. Front-end plain apache, and backend
                                      Message 18 of 22 , Feb 23, 2004
                                      • 0 Attachment
                                        Somehow a high volume site for your reference:

                                        System: P4 1.6G, 1G memory, Linux 7.2, Perl 5.6, customized kernel.

                                        Front-end plain apache, and backend mod_perl, + a local mysql database(!).

                                        The system currently handles 130K unique IP per day. At the peak time,
                                        the proxy serves typically 500 clients while the backend handles
                                        50 children. Normally, it is about 200 clients + 50 children.
                                        Keep-live is off. CPU usage is typically 70-90% full.

                                        Keep-live used to be ON, but then at the 9K IP daily hits, the whole
                                        system went down because the frontend quickly forked to 1024 (a limit
                                        I set) by too many instant requests. If that happend, the whole
                                        system would head to memory swap, hardly recovery.

                                        Generally speaking, the proxy + mod_perl fork model is quite scalable.


                                        Pod Merl

                                        On Mon, 2004-02-23 at 19:37, Richard F. Rebel wrote:
                                        > Eh well, do I get points for making a prod
                                        > system run with mp2 and mpm-worker?

                                        Certainly. We are all eager for this kind of info.

                                        > Most of our clients are *slow*, so perhaps this is why things seem to
                                        > work so well.

                                        Actually, if your clients are slow you would be better off with a
                                        reverse proxy to do buffering.

                                        > I have increased the number of interpreters and have
                                        > noticed a decrease in cpu usage and an improvement in performance in
                                        > general.

                                        Cool.

                                        > Now, does this sort of explain why I was seeing more memory usage with
                                        > prefork, slow clients?

                                        That was because you were running 500 interpreters instead of 50.

                                        > When exactly is the perl interpreter put back
                                        > into the 'free' list?

                                        More of a question for someone closer to the code than me, but I thought
                                        that the interpreter was tied up until the server finishes the request,
                                        meaning that slow clients will keep an interpreter tied up for a long
                                        time.

                                        - Perrin

                                        --
                                        Report problems: http://perl.apache.org/bugs/
                                        Mail list info: http://perl.apache.org/maillist/modperl.html
                                        List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                      • Stas Bekman
                                        ... Sounds like a job for Apache::Status. Though it will need an API to query the interpreter pools which AFAIK don t exist yet. Or may be it ll better suite
                                        Message 19 of 22 , Feb 23, 2004
                                        • 0 Attachment
                                          Richard F. Rebel wrote:

                                          > Speaking of, does anyone know of a way to tell exactly how many
                                          > interpreters are running in a given process?

                                          Sounds like a job for Apache::Status. Though it will need an API to query the
                                          interpreter pools which AFAIK don't exist yet. Or may be it'll better suite
                                          Apache::VMonitor http://search.cpan.org/dist/Apache-VMonitor/. At the moment
                                          it will show you all the active threads for each process, though it won't tell
                                          you which one is perl and which is not.

                                          Meanwhile if your interpreters pool is always growing and never get its items
                                          killed you can cheat and count the number of started interpreters by defining
                                          a function CLONE in some package, e.g. in httpd.conf:

                                          <Perl>
                                          package My::InterpreterCounter;
                                          use threads;
                                          use threads::shared;
                                          my $counter : shared = 0;
                                          sub CLONE {
                                          lock $counter;
                                          $counter++;
                                          }
                                          sub END {
                                          print STDERR "$counter interpreters were started";
                                          }
                                          </Perl>

                                          this is untested... the special function CLONE is called every time a new perl
                                          interpreter is created (via perl_clone()). It can exist in any package.

                                          __________________________________________________________________
                                          Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                                          http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                                          mailto:stas@... http://use.perl.org http://apacheweek.com
                                          http://modperlbook.org http://apache.org http://ticketmaster.com

                                          --
                                          Report problems: http://perl.apache.org/bugs/
                                          Mail list info: http://perl.apache.org/maillist/modperl.html
                                          List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                        • Richard F. Rebel
                                          ... Great hack Stas, but alas, I suspect(all right, I *know*) that there are interpreters that are getting reaped in my situation. Is there a similar magic
                                          Message 20 of 22 , Feb 23, 2004
                                          • 0 Attachment
                                            On Mon, 2004-02-23 at 20:34, Stas Bekman wrote:
                                            > Richard F. Rebel wrote:
                                            >
                                            > > Speaking of, does anyone know of a way to tell exactly how many
                                            > > interpreters are running in a given process?
                                            >
                                            > Sounds like a job for Apache::Status. Though it will need an API to query the
                                            > interpreter pools which AFAIK don't exist yet. Or may be it'll better suite
                                            > Apache::VMonitor http://search.cpan.org/dist/Apache-VMonitor/. At the moment
                                            > it will show you all the active threads for each process, though it won't tell
                                            > you which one is perl and which is not.
                                            >
                                            > Meanwhile if your interpreters pool is always growing and never get its items
                                            > killed you can cheat and count the number of started interpreters by defining
                                            > a function CLONE in some package, e.g. in httpd.conf:

                                            Great hack Stas, but alas, I suspect(all right, I *know*) that there are
                                            interpreters that are getting reaped in my situation. Is there a
                                            similar magic BLOCK for gc like the 'CLONE' convention?

                                            The CLONE special block should be documented somewhere, who shall I rag
                                            at for that??

                                            And if I( haven't said it b4, thanks for the hard work on mp2...

                                            > <Perl>
                                            > package My::InterpreterCounter;
                                            > use threads;
                                            > use threads::shared;
                                            > my $counter : shared = 0;
                                            > sub CLONE {
                                            > lock $counter;
                                            > $counter++;
                                            > }
                                            > sub END {
                                            > print STDERR "$counter interpreters were started";
                                            > }
                                            > </Perl>
                                            >
                                            > this is untested... the special function CLONE is called every time a new perl
                                            > interpreter is created (via perl_clone()). It can exist in any package.
                                            >
                                            > __________________________________________________________________
                                            > Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                                            > http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                                            > mailto:stas@... http://use.perl.org http://apacheweek.com
                                            > http://modperlbook.org http://apache.org http://ticketmaster.com
                                            --
                                            Richard F. Rebel <rrebel@...>
                                            WhenU.com
                                          • Stas Bekman
                                            ... I think the END block is run for every clone, I ll write a test later. Really, the API to query the status of interpreter pools would be the best. ... It s
                                            Message 21 of 22 , Feb 23, 2004
                                            • 0 Attachment
                                              Richard F. Rebel wrote:
                                              > On Mon, 2004-02-23 at 20:34, Stas Bekman wrote:
                                              >
                                              >>Richard F. Rebel wrote:
                                              >>
                                              >>
                                              >>>Speaking of, does anyone know of a way to tell exactly how many
                                              >>>interpreters are running in a given process?
                                              >>
                                              >>Sounds like a job for Apache::Status. Though it will need an API to query the
                                              >>interpreter pools which AFAIK don't exist yet. Or may be it'll better suite
                                              >>Apache::VMonitor http://search.cpan.org/dist/Apache-VMonitor/. At the moment
                                              >>it will show you all the active threads for each process, though it won't tell
                                              >>you which one is perl and which is not.
                                              >>
                                              >>Meanwhile if your interpreters pool is always growing and never get its items
                                              >>killed you can cheat and count the number of started interpreters by defining
                                              >>a function CLONE in some package, e.g. in httpd.conf:
                                              >
                                              >
                                              > Great hack Stas, but alas, I suspect(all right, I *know*) that there are
                                              > interpreters that are getting reaped in my situation. Is there a
                                              > similar magic BLOCK for gc like the 'CLONE' convention?

                                              I think the END block is run for every clone, I'll write a test later.

                                              Really, the API to query the status of interpreter pools would be the best.

                                              > The CLONE special block should be documented somewhere, who shall I rag
                                              > at for that??

                                              It's mentioned in perlmod:

                                              Ithreads work by cloning the data tree so that no data is shared
                                              between different threads. These threads can be used by using the
                                              "threads" module or by doing fork() on win32 (fake fork() support).
                                              When a thread is cloned all Perl data is cloned, however non-Perl data
                                              cannot be cloned automatically. Perl after 5.7.2 has support for the
                                              "CLONE" special subroutine . In "CLONE" you can do whatever you need
                                              to do, like for example handle the cloning of non-Perl data, if neces-
                                              sary. "CLONE" will be executed once for every package that has it
                                              defined (or inherits it). It will be called in the context of the new
                                              thread, so all modifications are made in the new area.

                                              If you want to CLONE all objects you will need to keep track of them
                                              per package. This is simply done using a hash and
                                              Scalar::Util::weaken().

                                              I'm working on a full blown example and docs of where it's indispensible:
                                              http://apache.org/~stas/Example-CLONE-0.02.tar.gz

                                              > And if I( haven't said it b4, thanks for the hard work on mp2...

                                              Thanks for using it! ;)
                                              __________________________________________________________________
                                              Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                                              http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                                              mailto:stas@... http://use.perl.org http://apacheweek.com
                                              http://modperlbook.org http://apache.org http://ticketmaster.com

                                              --
                                              Report problems: http://perl.apache.org/bugs/
                                              Mail list info: http://perl.apache.org/maillist/modperl.html
                                              List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                            • Stas Bekman
                                              Stas Bekman wrote: [...] ... I was wrong, it doesn t. It runs only by the main interpreter (I m talking pure perl here, it s not special to mod_perl). Here is
                                              Message 22 of 22 , Feb 24, 2004
                                              • 0 Attachment
                                                Stas Bekman wrote:
                                                [...]
                                                >> Great hack Stas, but alas, I suspect(all right, I *know*) that there are
                                                >> interpreters that are getting reaped in my situation. Is there a
                                                >> similar magic BLOCK for gc like the 'CLONE' convention?
                                                >
                                                >
                                                > I think the END block is run for every clone, I'll write a test later.

                                                I was wrong, it doesn't. It runs only by the main interpreter (I'm talking
                                                pure perl here, it's not special to mod_perl).

                                                Here is a package that will do the accounting for you. I've used a dummy
                                                object to use its DESTROY method to emulate END for cloned interpreters. All
                                                you need is to load this package at the server startup and look at your
                                                error_log during the server life and at its very end.

                                                Here is the package:

                                                # My/InterpreterCounter.pm
                                                package My::InterpreterCounter;

                                                use strict;
                                                use warnings FATAL => 'all';

                                                use threads;
                                                use threads::shared;

                                                use subs qw(say);

                                                # a special object created in the parent interpreter which will call
                                                # DESTROY when each interpreter goes down, providing the END
                                                # equivalent for spawned ithreads
                                                #
                                                # we also use it to track the thread id, while we have it
                                                my $obj = My::InterpreterCounter->new();

                                                my $ctr : shared = &share({});
                                                # 1 is the parent interpreter which already exists
                                                $ctr->{cnt} = 1;
                                                $ctr->{max} = 1;
                                                $ctr->{tot} = 1;

                                                sub new {
                                                my $class = shift;
                                                my $self = 0;
                                                return bless \$self, $class;
                                                }

                                                sub CLONE {
                                                my $tid = threads->self->tid;
                                                say "a cloned interpreter #$tid was spawned";
                                                $$obj = $tid;
                                                lock $ctr;
                                                $ctr->{tot}++;
                                                $ctr->{cnt}++;
                                                $ctr->{max}++ if $ctr->{cnt} > $ctr->{max};
                                                status();
                                                }

                                                sub DESTROY {
                                                my $self = shift;
                                                my $tid = $$self;
                                                lock $ctr;
                                                $ctr->{cnt}--;
                                                say "a cloned interpreter #$tid went down";
                                                status();
                                                }

                                                sub END {
                                                say "the main interpreter goes down";
                                                status();
                                                }

                                                sub status {
                                                lock $ctr;
                                                printf STDERR " " x 9 .
                                                "total: $ctr->{tot}, count $ctr->{cnt}, at most $ctr->{max}\n";
                                                }

                                                sub say {
                                                (my $caller = (caller(1))[3]) =~ s/.*:://;
                                                printf STDERR "%-7s: %s\n", $caller, join '', @_;
                                                }

                                                1;

                                                here is a standalone program that requires no mod_perl. As you can see it does
                                                nothing to the package besides loading it *before* it spawns any new threads:


                                                #test.pl
                                                use My::InterpreterCounter;

                                                use threads;
                                                use threads::shared;

                                                for (0..1) {
                                                my $thr1 = threads->new(\&worker);
                                                my $thr2 = threads->new(\&worker);
                                                $thr1->join;
                                                $thr2->join;
                                                }

                                                sub worker {
                                                my $tid = threads->self->tid;
                                                #print STDERR "TID is $tid\n";
                                                }

                                                Running it:

                                                % perl -I. test.pl
                                                CLONE : a cloned interpreter #1 was spawned
                                                total: 2, count 2, at most 2
                                                CLONE : a cloned interpreter #2 was spawned
                                                total: 3, count 3, at most 3
                                                DESTROY: a cloned interpreter #2 went down
                                                total: 3, count 2, at most 3
                                                DESTROY: a cloned interpreter #1 went down
                                                total: 3, count 1, at most 3
                                                CLONE : a cloned interpreter #3 was spawned
                                                total: 4, count 2, at most 3
                                                CLONE : a cloned interpreter #4 was spawned
                                                total: 5, count 3, at most 3
                                                DESTROY: a cloned interpreter #4 went down
                                                total: 5, count 2, at most 3
                                                DESTROY: a cloned interpreter #3 went down
                                                total: 5, count 1, at most 3
                                                END : the main interpreter goes down
                                                total: 5, count 1, at most 3
                                                DESTROY: a cloned interpreter #0 went down
                                                total: 5, count 0, at most 3

                                                So you can see that during the program life, at any given time there were at
                                                most 3 perl interpreters running (1 parent + 2 clones). And you can see that
                                                there were a total of 4 clones started (plus one parent perl).

                                                So I loaded this module from modperl-2.0/t/conf/modperl_extra.pl and run the
                                                threads tests (this is just a preforked mpm which spawn ithreads from the test):

                                                t/TEST -v perl/ithreads

                                                the error_log had:

                                                CLONE : a cloned interpreter #1 was spawned
                                                total: 2, count 2, at most 2
                                                Attempt to free unreferenced scalar: SV 0x9842900 during global destruction.
                                                DESTROY: a cloned interpreter #1 went down
                                                total: 2, count 1, at most 2
                                                CLONE : a cloned interpreter #2 was spawned
                                                total: 3, count 2, at most 2
                                                Attempt to free unreferenced scalar: SV 0x99de3f4 during global destruction.
                                                DESTROY: a cloned interpreter #2 went down
                                                total: 3, count 1, at most 2

                                                So, as you can see the test has spawned two clones, but only one was active at
                                                any given time. Indeed looking at modperl-2.0/t/response/TestPerl/ithreads.pm
                                                you can see that the first ithread was joined before a new one was spawned.

                                                Give it a try under a worker mpm and see if you get a nice report in a nice
                                                progress and the totals at the shutdown.

                                                __________________________________________________________________
                                                Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
                                                http://stason.org/ mod_perl Guide ---> http://perl.apache.org
                                                mailto:stas@... http://use.perl.org http://apacheweek.com
                                                http://modperlbook.org http://apache.org http://ticketmaster.com

                                                --
                                                Report problems: http://perl.apache.org/bugs/
                                                Mail list info: http://perl.apache.org/maillist/modperl.html
                                                List etiquette: http://perl.apache.org/maillist/email-etiquette.html
                                              Your message has been successfully submitted and would be delivered to recipients shortly.