Loading ...
Sorry, an error occurred while loading the content.

Re: Apache::ASP locking up

Expand Messages
  • Warren Young
    ... These systems are installed in schools, so they re only being used from roughly 7am to 4pm. This problem is happening within that window, while it is
    Message 1 of 10 , Nov 19, 2004
    • 0 Attachment
      Josh Chamas wrote:

      > What is the state of the system when it locks up?

      These systems are installed in schools, so they're only being used from
      roughly 7am to 4pm. This problem is happening within that window, while
      it is being used during the day. We have never seen it lock up in this
      way over night.

      > Is the machine under load

      To the extent that 5500 hits per day is "load", yes. :)

      There seems to be a loose correlation between the problem and load: this
      site is one of the heaviest users. We have heavier users, though, that
      aren't seeing the problem.

      > is a process spinning out of control,

      We don't notice a busied out CPU, if that's what you mean. And, I
      wouldn't think anything is truly deadlocked, since all it takes is
      removing the ASP state directory to fix the problem.

      > is there any errors in the error_log that look interesting?

      No, but then we send a lot of stuff to error_log ourselves. And, we
      don't have Debug turned up. Should we?

      > Does restarting the web server help things without removing the /tmp/asp
      > entirely,

      No.

      > or is removing that directory the only fix?

      Yes.

      > If the details
      > you provided about /tmp/asp are before removing but after a freeze,
      > then I would say there is nothing that looks out of the ordinary there
      > that would raise concern on my side.

      Hmmm... Would having the files help?

      ---------------------------------------------------------------------
      To unsubscribe, e-mail: asp-unsubscribe@...
      For additional commands, e-mail: asp-help@...
    • Fagyal Csongor
      Warren, Are you sure you don t put something wrong into $Application / $Session? Might it be that it is one of your ASP applications that freezes Apache,
      Message 2 of 10 , Nov 19, 2004
      • 0 Attachment
        Warren,

        Are you sure you don't put "something wrong" into $Application /
        $Session? Might it be that it is one of your ASP applications that
        freezes Apache, because of something you store in the above scopes?

        You might also try to write a small script that dumps your *full* perl
        namespace in mod_perl, to see if something is "leaking"...

        Regards,
        - Csongor

        >> What is the state of the system when it locks up?
        >
        >
        > These systems are installed in schools, so they're only being used
        > from roughly 7am to 4pm. This problem is happening within that
        > window, while it is being used during the day. We have never seen it
        > lock up in this way over night.
        >
        >> Is the machine under load
        >
        >
        > To the extent that 5500 hits per day is "load", yes. :)
        >
        > There seems to be a loose correlation between the problem and load:
        > this site is one of the heaviest users. We have heavier users,
        > though, that aren't seeing the problem.
        >
        >> is a process spinning out of control,
        >
        >
        > We don't notice a busied out CPU, if that's what you mean. And, I
        > wouldn't think anything is truly deadlocked, since all it takes is
        > removing the ASP state directory to fix the problem.
        >
        >> is there any errors in the error_log that look interesting?
        >
        >
        > No, but then we send a lot of stuff to error_log ourselves. And, we
        > don't have Debug turned up. Should we?
        >
        >> Does restarting the web server help things without removing the /tmp/asp
        >> entirely,
        >
        >
        > No.
        >
        >> or is removing that directory the only fix?
        >
        >
        > Yes.
        >
        >> If the details
        >> you provided about /tmp/asp are before removing but after a freeze,
        >> then I would say there is nothing that looks out of the ordinary there
        >> that would raise concern on my side.
        >
        >
        > Hmmm... Would having the files help?
        >


        ---------------------------------------------------------------------
        To unsubscribe, e-mail: asp-unsubscribe@...
        For additional commands, e-mail: asp-help@...
      • Warren Young
        ... Could be, but if so, why is only this site locking up? ... Do you remember the incantation for that? I m sure I can look it up if you don t have it off
        Message 3 of 10 , Nov 19, 2004
        • 0 Attachment
          Fagyal Csongor wrote:
          >
          > Are you sure you don't put "something wrong" into $Application /
          > $Session?

          Could be, but if so, why is only this site locking up?

          > You might also try to write a small script that dumps your *full* perl
          > namespace in mod_perl, to see if something is "leaking"...

          Do you remember the incantation for that? I'm sure I can look it up if
          you don't have it off the top of your head.

          ---------------------------------------------------------------------
          To unsubscribe, e-mail: asp-unsubscribe@...
          For additional commands, e-mail: asp-help@...
        • Christopher Hicks
          ... The db that s used for storing the sessions which seems to be the files you re deleting. There are a variety of dbm implementations and I m not sure which
          Message 4 of 10 , Nov 20, 2004
          • 0 Attachment
            On Fri, 19 Nov 2004, Warren Young wrote:
            > Christopher Hicks wrote:
            >> You might be tripping over a bug in the db code.
            >
            > A bug in which db code?

            The db that's used for storing the sessions which seems to be the files
            you're deleting. There are a variety of dbm implementations and I'm not
            sure which Apache::ASP ends up using on RH7.3.

            > If it's part of the open source underpinnings of Apache::ASP, why is the
            > bug still there? Why haven't you submitted a patch?

            The reason I suggested it might be a problem in the db code is that you
            could try recompiling that without much trouble. (Taking a more current
            rpm and rebuilding it for 7.3 would be the ideal solution, but if you're
            not familiar with tweaking the rpm build .spec files going from source
            would be easier.) I haven't run into any db problems with Apache::ASP,
            but we don't have any Red Hat 7, 8, or 9 boxes left in production at this
            point.

            >> Can you try the Apache::ASP on a Fedora Core 2 or 3 machine?
            >
            > We have no evidence that correlates the problem with platform. Before we
            > send a person out to a remote site to do an OS upgrade, we'd have to have a
            > pretty good idea that it will fix the problem.

            Considering that Red Hat 7.3 is long out of having patch support for it
            upgrading would seem to be a very good idea for reasons that have nothing
            to do with fixing this problem.

            >> try this out on a nonproduction box.)
            >
            > Since we don't know how the box is being locked up, how could we test
            > whether the OS upgrade helped?

            Try replicating the problem in a test environment?

            > Again, we have many, many other sites...surely if it was as simple as an
            > OS issue, some of our other RH7.3 systems would also be locking up.

            Not necessarily. If this site is your heaviest site then they may be
            tripping over a database locking issue that just doesn't have as good an
            odds of happening at he other sites.

            --
            </chris>

            "Fans of Mozilla's free, open-source Firefox browser make the
            ardent Apple faithful look like a bunch of slackers."
            - Rebecca Lieb at clickz.com

            ---------------------------------------------------------------------
            To unsubscribe, e-mail: asp-unsubscribe@...
            For additional commands, e-mail: asp-help@...
          • Josh Chamas
            Hi Warren, At this point, I would recommend setting Debug -1 and then stop/start apache, and send me error_log from start until freeze. Then stop apache, and
            Message 5 of 10 , Nov 21, 2004
            • 0 Attachment
              Hi Warren,

              At this point, I would recommend setting Debug -1 and then
              stop/start apache, and send me error_log from start until freeze.
              Then stop apache, and get me a tar/gzip copy of the directory /tmp/asp
              *unless* there is sensitive data in the data structures in
              which case you should not be sending me that.

              It may be something that is being stored in $Session or $Application
              that is killing things, and is only specific to that DBM format.

              Also, if all this debugging is just taking too long, then you
              might do something like this just for this one server:

              # httpd.conf
              <Perl>
              use Apache::ASP::StateManager;
              $Apache::ASP::State::DefaultStateDB = 'DB_File';
              </Perl>
              PerlSetVar StateDB DB_File
              # end httpd.conf

              By default Apache::ASP uses SDBM_File for the internal database
              storage. You can also try the new PerlSetVar without the <Perl>
              block. You can also try GDBM_File without using DB_File, either
              way as you may only have one or the other installed on your system.

              Note these things in <Perl> blocks in not officially supported,
              but this is a worst case scenario where SDBM_File just isn't working
              for you for the internal database, and I think this should work to
              override it. Note also, if you use the <Perl> method, you need
              to delete the /tmp/asp directory between changing the DefaultStateDB
              setting, as Apache::ASP is not used to this being changed at runtime,
              its a hard coded global in the code.

              Regards,

              Josh


              Warren Young wrote:
              > Josh Chamas wrote:
              >
              >> What is the state of the system when it locks up?
              >
              >
              > These systems are installed in schools, so they're only being used from
              > roughly 7am to 4pm. This problem is happening within that window, while
              > it is being used during the day. We have never seen it lock up in this
              > way over night.
              >
              >> Is the machine under load
              >
              >
              > To the extent that 5500 hits per day is "load", yes. :)
              >
              > There seems to be a loose correlation between the problem and load: this
              > site is one of the heaviest users. We have heavier users, though, that
              > aren't seeing the problem.
              >
              >> is a process spinning out of control,
              >
              >
              > We don't notice a busied out CPU, if that's what you mean. And, I
              > wouldn't think anything is truly deadlocked, since all it takes is
              > removing the ASP state directory to fix the problem.
              >
              >> is there any errors in the error_log that look interesting?
              >
              >
              > No, but then we send a lot of stuff to error_log ourselves. And, we
              > don't have Debug turned up. Should we?
              >
              >> Does restarting the web server help things without removing the /tmp/asp
              >> entirely,
              >
              >
              > No.
              >
              >> or is removing that directory the only fix?
              >
              >
              > Yes.
              >
              >> If the details
              >> you provided about /tmp/asp are before removing but after a freeze,
              >> then I would say there is nothing that looks out of the ordinary there
              >> that would raise concern on my side.
              >
              >
              > Hmmm... Would having the files help?
              >
              > ---------------------------------------------------------------------
              > To unsubscribe, e-mail: asp-unsubscribe@...
              > For additional commands, e-mail: asp-help@...
              >
              >

              ---------------------------------------------------------------------
              To unsubscribe, e-mail: asp-unsubscribe@...
              For additional commands, e-mail: asp-help@...
            • Warren Young
              ... Okay, I ll do that. School is out this week at the problem site, so I don t expect it to die this week. ... It does slow things down, but we can tolerate
              Message 6 of 10 , Nov 22, 2004
              • 0 Attachment
                Josh Chamas wrote:
                >
                > At this point, I would recommend setting Debug -1 and then

                Okay, I'll do that. School is out this week at the problem site, so I
                don't expect it to die this week.

                > Also, if all this debugging is just taking too long, then you
                > might do something like this just for this one server:

                It does slow things down, but we can tolerate it.

                Thanks!

                ---------------------------------------------------------------------
                To unsubscribe, e-mail: asp-unsubscribe@...
                For additional commands, e-mail: asp-help@...
              Your message has been successfully submitted and would be delivered to recipients shortly.