Loading ...
Sorry, an error occurred while loading the content.

mp2 + worker mpm + threads + threads::shared + PerlChildInitHandler

Expand Messages
  • Richard F. Rebel
    Hello, I have a high volume application that basically needs to keep track of a couple counters for the number of times certain requests were made and update a
    Message 1 of 4 , Jan 17, 2005
    • 0 Attachment
      Hello,

      I have a high volume application that basically needs to keep track of a
      couple counters for the number of times certain requests were made and
      update a MySQL db.

      Unfortunately, it's high volume enough that it's no longer possible to
      keep these counters in the databases updated in real time. (updates are
      to the order of 1000's per second).

      So, my goal is to alter my module so that upon the fork of each child,
      with X number of perl interp threads, to create two things:

      * A shared hash to keep track of all of the counters, so that interp
      threads can lock() and update as neccessary.

      * A overseer/manager thread that wakes up once every so often and
      updates the MySQL database with the contents of the global shared hash.

      I have the first one handled fine. All the interpreter threads in a
      process see the same data, increments work fine.

      The second part has me stumped. I use Apache->server->push_handlers
      called from a BEGIN block to install a PerlChildInitHandler, and this
      subroutine starts a detached thread that executes the code to sleep and
      update the datbase.

      Well, it works, but the only 'manager' thread that seems to start is for
      the main apache process (which doesn't handle requests directly
      anyways). In the children this never executes. There is an error in
      the apache log that comes immediately upon server startup complaining
      that a thread had exited while others were running. I suspect this
      error is due to apache daemonizing on startup.

      Where is the correct place to start such an 'manager' thread in each
      apache child? I thought that by installing a ChildInitHandler from the
      BEGIN block would work just fine...

      Thanks!



      --
      Richard F. Rebel

      cat /dev/null > `tty`
    • Richard F. Rebel
      Another good idea... :) But I am transfixed by this problem... I can t seem to get each forked apache server to have both a shared global hash between all
      Message 2 of 4 , Jan 17, 2005
      • 0 Attachment
        Another good idea... :)

        But I am transfixed by this problem... I can't seem to get each forked
        apache server to have both a shared global hash between all cloned
        interpreters, *and* one thread in each process that runs in the
        background doing housekeeping. I can think of numerous things that this
        would be useful for.

        I know I am close, but I can't seem to quite grasp what I am missing. I
        thought PerlChildInit's were called for each forked child from it's
        first/main interpreter (the one that all the others are cloned from).


        On Mon, 2005-01-17 at 13:59 -0500, Perrin Harkins wrote:
        > On Mon, 2005-01-17 at 11:25 -0500, Richard F. Rebel wrote:
        > > Unfortunately, it's high volume enough that it's no longer possible to
        > > keep these counters in the databases updated in real time. (updates are
        > > to the order of 1000's per second).
        >
        > I would just use BerkeleyDB for this, which can easilly keep up, rather
        > than messing with threads, but I'm interested in seeing if your
        > threading idea will work well.
        >
        > > * A overseer/manager thread that wakes up once every so often and
        > > updates the MySQL database with the contents of the global shared hash.
        >
        > Rather than doing that, why not just update it from a cleanup handler
        > every time the counter goes up by 10000 or so? Seems much easier to me.
        >
        > - Perrin
        >
        --
        Richard F. Rebel

        cat /dev/null > `tty`
      • Perrin Harkins
        ... I would just use BerkeleyDB for this, which can easilly keep up, rather than messing with threads, but I m interested in seeing if your threading idea will
        Message 3 of 4 , Jan 17, 2005
        • 0 Attachment
          On Mon, 2005-01-17 at 11:25 -0500, Richard F. Rebel wrote:
          > Unfortunately, it's high volume enough that it's no longer possible to
          > keep these counters in the databases updated in real time. (updates are
          > to the order of 1000's per second).

          I would just use BerkeleyDB for this, which can easilly keep up, rather
          than messing with threads, but I'm interested in seeing if your
          threading idea will work well.

          > * A overseer/manager thread that wakes up once every so often and
          > updates the MySQL database with the contents of the global shared hash.

          Rather than doing that, why not just update it from a cleanup handler
          every time the counter goes up by 10000 or so? Seems much easier to me.

          - Perrin
        • Richard F. Rebel
          Hi, Well, the problem was my fault. :/ I had a bug in a generic base class I use that makes it easier to build classes that work both inside and outside
          Message 4 of 4 , Jan 17, 2005
          • 0 Attachment
            Hi,

            Well, the problem was my fault. :/ I had a bug in a generic base class
            I use that makes it easier to build classes that work both inside and
            outside mod_perl.

            For those of you who are interested, this solutions works well. By
            using PerlChildInit handler to create a thread to maintain a shared
            global hash of hashes with some small portion of a database, and some
            information acquired with XML::RPC, my systems perform much better.

            We were having problems with availability and performance of the
            external data sources (eg MySQL or remote XML::RPC servers) that would
            cause our apache instances to wait around timing out for each request.

            Yay.

            Here is a demonstration module for those of interest:

            package TestPerlChildInit;

            use strict;

            use lib '/opt/whenu/lib/whenu-perl-lib';

            use threads;
            use threads::shared;

            use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS);
            use vars qw($DEBUG *DBG $CFG);
            use vars qw(%SHARED);

            @ISA = qw(Exporter);
            @EXPORT = qw();
            @EXPORT_OK = qw();
            %EXPORT_TAGS = (':DEFAULT' => [qw()],
            ':handler' => [qw()]);


            BEGIN {
            use mod_perl;
            use Apache2;
            use Apache::Const qw(:common :http);
            use Apache::RequestRec qw();
            use Apache::RequestIO qw();
            use Apache::Connection qw();
            use Apache::ServerUtil qw();
            use Apache::Module qw();
            use Apache::Util qw();
            use Apache::URI qw();
            use Apache::Log qw();
            use APR::OS qw();
            use APR::Table qw();
            share(%SHARED);
            $SHARED{'test'} = &share({});
            $SHARED{'test'}->{'count'} = 1;
            my $res = Apache->server->push_handlers(PerlChildInitHandler => \&mod_perl_ChildInitHandler);
            print STDERR "Testing[$$]: Installed ChildInitHandler result '$res'\n";
            }

            sub mod_perl_ChildInitHandler {
            print STDERR "mod_perl_ChildInitHandler\n";
            ## Start a thread to restart other thread...
            threads->new( sub {
            while(1) {
            my $ovs = threads->new(\&overseer);
            print STDERR "Testing[$$]: Started overseer thread\n";
            $ovs->join();
            print STDERR "Testing[$$]: Joined overseer thread (probably bad)\n";
            ## Add backoff for spawning too quickly etc.
            }
            })->detach;
            return &Apache::OK;
            }

            sub overseer {
            print STDERR "Testing[$$]->", threads->self->tid, " Overseer Startup...\n";
            while(sleep 2) {
            lock(%{$SHARED{'test'}});
            print STDERR "Testing[$$]->", threads->self->tid, ": \$SHARED{'test'}->{'count'} = $SHARED{'test'}->{'count'} \n";
            ## here is where you can do more interesting things such as
            ## get data from databases, external sources, or update them.
            }
            }

            sub handler : method {
            my $class = shift;
            my $r = shift;
            lock(%{$SHARED{'test'}});
            $SHARED{'test'}->{'count'}++;
            $r->no_cache();
            $r->err_headers_out->{"Expires"} = "Sat, 1 Jan 2000 00:00:00 GMT";
            $r->err_headers_out->{"Pragma"} = "no-cache";
            $r->err_headers_out->{"Cache-Control"} = "no-cache";
            $r->err_headers_out->{"Location"} = 'http://www.google.com';
            $r->status(&Apache::REDIRECT);
            $r->rflush();
            return &Apache::OK;

            }


            On Mon, 2005-01-17 at 13:59 -0500, Richard F. Rebel wrote:
            > Another good idea... :)
            >
            > But I am transfixed by this problem... I can't seem to get each forked
            > apache server to have both a shared global hash between all cloned
            > interpreters, *and* one thread in each process that runs in the
            > background doing housekeeping. I can think of numerous things that this
            > would be useful for.
            >
            > I know I am close, but I can't seem to quite grasp what I am missing. I
            > thought PerlChildInit's were called for each forked child from it's
            > first/main interpreter (the one that all the others are cloned from).
            >
            >
            > On Mon, 2005-01-17 at 13:59 -0500, Perrin Harkins wrote:
            > > On Mon, 2005-01-17 at 11:25 -0500, Richard F. Rebel wrote:
            > > > Unfortunately, it's high volume enough that it's no longer possible to
            > > > keep these counters in the databases updated in real time. (updates are
            > > > to the order of 1000's per second).
            > >
            > > I would just use BerkeleyDB for this, which can easilly keep up, rather
            > > than messing with threads, but I'm interested in seeing if your
            > > threading idea will work well.
            > >
            > > > * A overseer/manager thread that wakes up once every so often and
            > > > updates the MySQL database with the contents of the global shared hash.
            > >
            > > Rather than doing that, why not just update it from a cleanup handler
            > > every time the counter goes up by 10000 or so? Seems much easier to me.
            > >
            > > - Perrin
            > >
            --
            Richard F. Rebel

            cat /dev/null > `tty`
          Your message has been successfully submitted and would be delivered to recipients shortly.