Loading ...
Sorry, an error occurred while loading the content.

Problems with the State Manager

Expand Messages
  • Mark T. Dame
    Hello all. We have a strange problem with one of our web sites that uses Apache::ASP. Every week or so it starts slowing down to a crawl. If we clear out the
    Message 1 of 4 , Sep 17, 2002
    • 0 Attachment
      Hello all.

      We have a strange problem with one of our web sites that uses
      Apache::ASP. Every week or so it starts slowing down to a crawl. If we
      clear out the StateDir, it returns to normal. It almost seems like
      Apache::ASP isn't cleaning up it's state files under some circumstance
      (although what those are is still a mystery).

      We have also noticed errors like this:

      [Mon Sep 16 21:06:48 2002] [error] Can't use string
      ("ile','refresh_timeout' =>
      '7200'") as a HASH ref while "strict refs" in use at
      /usr/lib/perl5/site_perl/5.
      6.0/Apache/ASP/StateManager.pm line 232.

      We are currently using Apache::ASP version 2.39. We have StateDB set to
      DB_File. (Apache 1.3.26, mod_perl 1.27, perl 5.6.0)

      Any ideas?


      -m
      --
      ## Mark T. Dame, Vice President, Internet Operations
      ## MFM Communication Software: http://www.mfm.com/
      ## E-mail: mailto:mdame@... WWW: http://www.mfm.com/~mdame/
      "Yesterday I heard a tree muttering how it wished the HP4 would go
      *up* [in price]."
      -- Daniel Mocsny

      ---------------------------------------------------------------------
      To unsubscribe, e-mail: asp-unsubscribe@...
      For additional commands, e-mail: asp-help@...
    • Josh Chamas
      ... Looks like some kind of data corruption for the internal state database that would prevent Apache::ASP from cleaning up the state files correctly. Do you
      Message 2 of 4 , Sep 17, 2002
      • 0 Attachment
        Mark T. Dame wrote:
        > Hello all.
        >
        > We have a strange problem with one of our web sites that uses
        > Apache::ASP. Every week or so it starts slowing down to a crawl. If we
        > clear out the StateDir, it returns to normal. It almost seems like
        > Apache::ASP isn't cleaning up it's state files under some circumstance
        > (although what those are is still a mystery).
        >
        > We have also noticed errors like this:
        >
        > [Mon Sep 16 21:06:48 2002] [error] Can't use string
        > ("ile','refresh_timeout' =>
        > '7200'") as a HASH ref while "strict refs" in use at
        > /usr/lib/perl5/site_perl/5.
        > 6.0/Apache/ASP/StateManager.pm line 232.
        >
        > We are currently using Apache::ASP version 2.39. We have StateDB set to
        > DB_File. (Apache 1.3.26, mod_perl 1.27, perl 5.6.0)
        >

        Looks like some kind of data corruption for the internal
        state database that would prevent Apache::ASP from cleaning
        up the state files correctly. Do you have the StateDir shared between
        hosts on some network file system? This could cause corruption.
        Failing that, you may have uncovered a scalability issue
        with SDBM_File which is used for the internal state dbm always
        for speed.

        If you want to change the type of dbm used for the internal
        database, you can do this:

        PerlModule Apache::ASP
        <Perl>
        $Apache::ASP::DefaultStateDB = 'DB_File';
        </Perl>

        StateDB only works on $Application & $Session, but not
        the internal database, so I believe the above will work.

        I could also add some code that could work around the
        data corruption issue you are facing, and we can try
        that if you like.

        Regards,

        Josh
        ________________________________________________________________
        Josh Chamas, Founder phone:925-552-0128
        Chamas Enterprises Inc. http://www.chamas.com
        NodeWorks Link Checking http://www.nodeworks.com


        ---------------------------------------------------------------------
        To unsubscribe, e-mail: asp-unsubscribe@...
        For additional commands, e-mail: asp-help@...
      • Mark T. Dame
        ... Yeah. I forgot to mention that part. I have all of my web directories (including the state directories) on NFS mounts since any given request can be
        Message 3 of 4 , Sep 17, 2002
        • 0 Attachment
          Josh Chamas wrote:
          >
          > Mark T. Dame wrote:
          > > Hello all.
          > >
          > > We have a strange problem with one of our web sites that uses
          > > Apache::ASP. Every week or so it starts slowing down to a crawl. If we
          > > clear out the StateDir, it returns to normal. It almost seems like
          > > Apache::ASP isn't cleaning up it's state files under some circumstance
          > > (although what those are is still a mystery).
          > >
          > > We have also noticed errors like this:
          > >
          > > [Mon Sep 16 21:06:48 2002] [error] Can't use string
          > > ("ile','refresh_timeout' =>
          > > '7200'") as a HASH ref while "strict refs" in use at
          > > /usr/lib/perl5/site_perl/5.
          > > 6.0/Apache/ASP/StateManager.pm line 232.
          > >
          > > We are currently using Apache::ASP version 2.39. We have StateDB set to
          > > DB_File. (Apache 1.3.26, mod_perl 1.27, perl 5.6.0)
          > >
          >
          > Looks like some kind of data corruption for the internal
          > state database that would prevent Apache::ASP from cleaning
          > up the state files correctly. Do you have the StateDir shared between
          > hosts on some network file system?

          Yeah. I forgot to mention that part. I have all of my web directories
          (including the state directories) on NFS mounts since any given request
          can be handled by one of a number of physical servers.


          > This could cause corruption.

          So SDBM_File doesn't handle NFS well?


          > Failing that, you may have uncovered a scalability issue
          > with SDBM_File which is used for the internal state dbm always
          > for speed.
          >
          > If you want to change the type of dbm used for the internal
          > database, you can do this:
          >
          > PerlModule Apache::ASP
          > <Perl>
          > $Apache::ASP::DefaultStateDB = 'DB_File';
          > </Perl>
          >
          > StateDB only works on $Application & $Session, but not
          > the internal database, so I believe the above will work.

          I'll give this a shot and see what happens.


          > I could also add some code that could work around the
          > data corruption issue you are facing, and we can try
          > that if you like.

          We'll try the different internal state DB first. I'll let you know what
          happens.


          -m
          --
          ## Mark T. Dame, Vice President, Internet Operations
          ## MFM Communication Software: http://www.mfm.com/
          ## E-mail: mailto:mdame@... WWW: http://www.mfm.com/~mdame/
          "Where a calculator on the ENIAC is equipped with 18,000 vacuum
          tubes and weighs 30 tons, computers in the future may have only
          1,000 vacuum tubes and perhaps weigh 1 1/2 tons."
          -- Popular Mechanics, March 1949

          ---------------------------------------------------------------------
          To unsubscribe, e-mail: asp-unsubscribe@...
          For additional commands, e-mail: asp-help@...
        • Josh Chamas
          ... Keep SDBM_File then, as its the most NFS corruption tolerant. You need to make sure that your NFS clients & servers are supporting flock() correctly over
          Message 4 of 4 , Sep 17, 2002
          • 0 Attachment
            Mark T. Dame wrote:
            >
            >
            > Yeah. I forgot to mention that part. I have all of my web directories
            > (including the state directories) on NFS mounts since any given request
            > can be handled by one of a number of physical servers.
            >

            Keep SDBM_File then, as its the most NFS corruption tolerant.
            You need to make sure that your NFS clients & servers are supporting
            flock() correctly over the network, or it really not safe to run the
            StateDir in this way, though it mostly works as you have found.
            Also, make sure your NFS clients never cache or buffer the data,
            then you will also have data sync issues.

            To make sure your NFS are locking correctly, try this test,
            common StateDir over NFS between 2 servers. Have an ASP
            script write something like:

            <%= sprintf("%06d", $Application->{Count}++ )%>

            Then hit the same script on 2 different servers with ab
            for something like 1000 requests. The final result should
            be something near 2001 when you view it in a browser, if
            its not, you do not have flock() locking supported
            across the network. I believe old NFS v2 did not do this
            well, but that NFS v3 might, but have no real world experience
            to back this up, just 2nd hand info.

            If you want to continue with a StateDir approach, & NFS
            does not get locking to work, then consider a samba mount.
            Samba does support flock() locking on access to a network share.
            If you have a NetApp, this might be a CIFS share instead.

            If you want to abandon StateDir entirely, it is trivial to
            wire up a custom $Session from an Apache::Session in Script_OnStart
            for your application. What you gain in RDBMS backed sessions,
            but you lose the session management & expiration.

            Regards,

            Josh
            ________________________________________________________________
            Josh Chamas, Founder phone:925-552-0128
            Chamas Enterprises Inc. http://www.chamas.com
            NodeWorks Link Checking http://www.nodeworks.com


            ---------------------------------------------------------------------
            To unsubscribe, e-mail: asp-unsubscribe@...
            For additional commands, e-mail: asp-help@...
          Your message has been successfully submitted and would be delivered to recipients shortly.