Loading ...
Sorry, an error occurred while loading the content.

tuxedo error when monitoring an application server domain.

Expand Messages
  • tdcornwell_120maple
    I have noticed that running a command line invocation of tmadmin on a running domain will *sometimes* generate an error: LIBTUX_CAT:577: Unable to register
    Message 1 of 4 , Feb 26, 2013
    • 0 Attachment
      I have noticed that running a command line invocation of tmadmin on a running domain will *sometimes* generate an error:

      "LIBTUX_CAT:577: Unable to register because the slot is already owned by another process" I can get this by running something as simple as:
      "psadmin -c [sstatus, or cstatus, or qstatus] -d mydomain"

      I have tracked this down to an ORA doc [ID 764563.1] but the description isn't very encouraging:

      "...Such an error is reported by a TUXEDO administrative process that is trying to register in the Bulletin Board. For
      recovery purposes, some TUXEDO administrative processes (restartsrv, cleanupsrv, ...) use a reserved slot in the BB.
      So running multiple instances of the same process may cause the first instance to register using the reserved slot and
      subsequent instances to report LIBTUX_CAT:577: ERROR: Unable to register because the slot is already owned by another
      process...."

      The reason I'm concerned is because, if I run some of these command line invocations to gather monitoring information on the domain, and wonder if this actually can cause the running instance to fail?

      I know that on occasion, starting the domain fails, and I'm pretty sure that the monitoring task is the cause.

      Is there a way to avoid this?

      Thanks,
      -T
    • David Kurtz
      By coincidence, one of my customers has been experiencing the same problem. It is not completely clear what is the exact cause. There is some suggestion that
      Message 2 of 4 , Mar 1 9:52 AM
      • 0 Attachment
        RE: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

        By coincidence, one of my customers has been experiencing the same problem.  It is not completely clear what is the exact cause.  There is some suggestion that LIBTUX_CAT:577 is a symptom of a corruption in the bulletin board but I have no direct evidence for this.   

        In our case we were running Tuxedo 9.1 with the base rolling patch - level 036.  Oracle support recommended applying rolling patch 095 - and there are various support notes that support this recommendation.  We have done so and are now waiting for the next peak processing day when we have had the errors.

        I don't completely agree with note 764563.1 - only one instance of the restartsrv can exist in a single domain, so there is never concurrent recycling.  Recycling of one server can be deferred while another is recycled.  However, very frequent recycling in a busy domain may be able produce these conditions.

        Psadmin takes a lock out on the bulletin board.  If you are writing scripts that use tmadmin to interrogate the BB and collect metrics then use tmadmin -r because it doesn't lock the BB in administrative mode.  If you open tmadin with the -r option twice you may see.

        TMADMIN_CAT:199: WARN: Cannot become administrator.Limited set of commands available.

        I can also reproduce the LIBTUX_CAT:577 error during recycling, but this is generated by the tmadmin process itself, rather than by BBL process as we are experiencing.

        230809.GO-FASTER-6!tmadmin.6652.2800.-2: LIBTUX_CAT:577: ERROR: Unable to register because the slot is already owned by another process

        230821.GO-FASTER-6!tmadmin.7084.7852.-2: 02-26-2013: Tuxedo Version 10.3.0.0 with VS2008, 32-bit

        230821.GO-FASTER-6!tmadmin.7084.7852.-2: TMADMIN_CAT:1330: INFO: Command: stop -i 2

        231013.GO-FASTER-6!PSAPPSRV.6528.5656.0: LIBTUX_CAT:1393: WARN: tpreturn called with TPEXIT

        231014.GO-FASTER-6!JSH.7200.7204.-2: JOLT_CAT:1043: "ERROR: tpacall() call failed, tperrno = 6"

        231014.GO-FASTER-6!JSH.7200.7204.-2: JOLT_CAT:1043: "ERROR: tpacall() call failed, tperrno = 6"

        regards

        _________________________

        David Kurtz

        tel: +44 (0)7771 760660

        mailto:david.kurtz@...



        -----Original Message-----
        From:
        psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of tdcornwell_120maple
        Sent: 26 February 2013 22:20
        To:
        psftdba@yahoogroups.com
        Subject: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

        I have noticed that running a command line invocation of tmadmin on a running domain will *sometimes* generate an error:

        "LIBTUX_CAT:577: Unable to register because the slot is already owned by another process"  I can get this by running something as simple as:

         "psadmin -c [sstatus, or cstatus, or qstatus] -d mydomain"

        I have tracked this down to an ORA doc [ID 764563.1] but the description isn't very encouraging:

        "...Such an error is reported by a TUXEDO administrative process that is trying to register in the Bulletin Board. For recovery purposes, some TUXEDO administrative processes (restartsrv, cleanupsrv, ...) use a reserved slot in the BB.

        So running multiple instances of the same process may cause the first instance to register using the reserved slot and subsequent instances to report LIBTUX_CAT:577: ERROR: Unable to register because the slot is already owned by another process...."

        The reason I'm concerned is because, if I run some of these command line invocations to gather monitoring information on the domain, and wonder if this actually can cause the running instance to fail?

        I know that on occasion, starting the domain fails, and I'm pretty sure that the monitoring task is the cause.

        Is there a way to avoid this?

        Thanks,

        -T




      • Juarez, Jose Antonio
        Sometime ago I have a similar problem, and I upgraded to the same Rolling Patch that say David Kurtz. Since then I begin to use tmadmin instead of psadmin to
        Message 3 of 4 , Mar 1 11:06 AM
        • 0 Attachment

          Sometime ago I have a similar problem, and I upgraded to the same Rolling Patch that say David Kurtz.

          Since then I begin  to use tmadmin instead of psadmin to monitor Application Server and Process Scheduler.

           

          This is my script that I have to monitoring services in a cron.

          -------------------------------------------------------

          #!/bin/ksh

          # JAJUPA For monitoring Tuxedo services.

           

          export TUXOFFSET=0

          FechaMon=`date "+%y%m%d"`

          FechaHoy=`date "+%y/%m/%d %H:%M:%S"`

           

          # --------------------------------------------------------------------------------------------------------------------------------

          # Inicia el procedimiento para Monitorear el NOMPRREG -

          # --------------------------------------------------------------------------------------------------------------------------------

          . /psdb/PT8.49/psconfigNOMPRREG.sh

           

          echo Inicia el procedimiento para Monitorear Servicios de Tuxedo >  /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo Application Server NOMPRREG >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

           

          export TUXCONFIG=/psdb/PT8.49/appserv/NOMPRREG/PSTUXCFG

          /psdb/bea/tuxedo91/bin/tmadmin < /psdb/utilerias/monitoreotuxedo/comandostmadmin.in >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

           

          echo >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo Process Scheduler NOMPRREG >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          export TUXCONFIG=/psdb/PT8.49/appserv/prcs/NOMPRREG/PSTUXCFG

          /psdb/bea/tuxedo91/bin/tmadmin < /psdb/utilerias/monitoreotuxedo/comandostmadmin.in >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

           

          # Begin the code for send email the file /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          #...

          -------------------------------------------------------

           

          And the file comandostmadmin.in

          -------------------------------------------------------

          psr

          psc

          pclt

          pq

          q

          -------------------------------------------------------

           

           

           

          From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of David Kurtz
          Sent: viernes, 01 de marzo de 2013 11:52 a.m.
          To: psftdba@yahoogroups.com
          Subject: RE: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

           

           

          By coincidence, one of my customers has been experiencing the same problem.  It is not completely clear what is the exact cause.  There is some suggestion that LIBTUX_CAT:577 is a symptom of a corruption in the bulletin board but I have no direct evidence for this.   

          In our case we were running Tuxedo 9.1 with the base rolling patch - level 036.  Oracle support recommended applying rolling patch 095 - and there are various support notes that support this recommendation.  We have done so and are now waiting for the next peak processing day when we have had the errors.

          I don't completely agree with note 764563.1 - only one instance of the restartsrv can exist in a single domain, so there is never concurrent recycling.  Recycling of one server can be deferred while another is recycled.  However, very frequent recycling in a busy domain may be able produce these conditions.

          Psadmin takes a lock out on the bulletin board.  If you are writing scripts that use tmadmin to interrogate the BB and collect metrics then use tmadmin -r because it doesn't lock the BB in administrative mode.  If you open tmadin with the -r option twice you may see.

          TMADMIN_CAT:199: WARN: Cannot become administrator.Limited set of commands available.

          I can also reproduce the LIBTUX_CAT:577 error during recycling, but this is generated by the tmadmin process itself, rather than by BBL process as we are experiencing.

          230809.GO-FASTER-6!tmadmin.6652.2800.-2: LIBTUX_CAT:577: ERROR: Unable to register because the slot is already owned by another process

          230821.GO-FASTER-6!tmadmin.7084.7852.-2: 02-26-2013: Tuxedo Version 10.3.0.0 with VS2008, 32-bit

          230821.GO-FASTER-6!tmadmin.7084.7852.-2: TMADMIN_CAT:1330: INFO: Command: stop -i 2

          231013.GO-FASTER-6!PSAPPSRV.6528.5656.0: LIBTUX_CAT:1393: WARN: tpreturn called with TPEXIT

          231014.GO-FASTER-6!JSH.7200.7204.-2: JOLT_CAT:1043: "ERROR: tpacall() call failed, tperrno = 6"

          231014.GO-FASTER-6!JSH.7200.7204.-2: JOLT_CAT:1043: "ERROR: tpacall() call failed, tperrno = 6"

          regards

          _________________________

          David Kurtz

          tel: +44 (0)7771 760660

          mailto:david.kurtz@...

           

          -----Original Message-----
          From:
          psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of tdcornwell_120maple
          Sent: 26 February 2013 22:20
          To:
          psftdba@yahoogroups.com
          Subject: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

          I have noticed that running a command line invocation of tmadmin on a running domain will *sometimes* generate an error:

          "LIBTUX_CAT:577: Unable to register because the slot is already owned by another process"  I can get this by running something as simple as:

           "psadmin -c [sstatus, or cstatus, or qstatus] -d mydomain"

          I have tracked this down to an ORA doc [ID 764563.1] but the description isn't very encouraging:

          "...Such an error is reported by a TUXEDO administrative process that is trying to register in the Bulletin Board. For recovery purposes, some TUXEDO administrative processes (restartsrv, cleanupsrv, ...) use a reserved slot in the BB.

          So running multiple instances of the same process may cause the first instance to register using the reserved slot and subsequent instances to report LIBTUX_CAT:577: ERROR: Unable to register because the slot is already owned by another process...."

          The reason I'm concerned is because, if I run some of these command line invocations to gather monitoring information on the domain, and wonder if this actually can cause the running instance to fail?

          I know that on occasion, starting the domain fails, and I'm pretty sure that the monitoring task is the cause.

          Is there a way to avoid this?

          Thanks,

          -T



        • David Kurtz
          If you are going to do that, invoke tmadmin with the -r option. As, example, I do in my tuxmon scripts (http://www.go-faster.co.uk/scripts/tuxmon.zip) Thus .
          Message 4 of 4 , Mar 1 11:37 AM
          • 0 Attachment

            If you are going to do that, invoke tmadmin with the –r option.  As, example, I do in my tuxmon scripts (http://www.go-faster.co.uk/scripts/tuxmon.zip)

            Thus

             

            export TUXCONFIG=/psdb/PT8.49/appserv/NOMPRREG/PSTUXCFG

            /psdb/bea/tuxedo91/bin/tmadmin –r < /psdb/utilerias/monitoreotuxedo/comandostmadmin.in >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

             

            echo >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

            echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

            echo Process Scheduler NOMPRREG >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

            echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

            export TUXCONFIG=/psdb/PT8.49/appserv/prcs/NOMPRREG/PSTUXCFG

            /psdb/bea/tuxedo91/bin/tmadmin –r < /psdb/utilerias/monitoreotuxedo/comandostmadmin.in >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

             

            regards
            _________________________
            David Kurtz
            tel: +44 (0)7771 760660
            mailto:david.kurtz@...

             

             

            From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of Juarez, Jose Antonio
            Sent: 01 March 2013 19:07
            To: psftdba@yahoogroups.com
            Subject: RE: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

             




            Sometime ago I have a similar problem, and I upgraded to the same Rolling Patch that say David Kurtz.

            Since then I begin  to use tmadmin instead of psadmin to monitor Application Server and Process Scheduler.

             

            This is my script that I have to monitoring services in a cron.

            -------------------------------------------------------

            #!/bin/ksh

            # JAJUPA For monitoring Tuxedo services.

             

            export TUXOFFSET=0

            FechaMon=`date "+%y%m%d"`

            FechaHoy=`date "+%y/%m/%d %H:%M:%S"`

             

            # --------------------------------------------------------------------------------------------------------------------------------

            # Inicia el procedimiento para Monitorear el NOMPRREG -

            # --------------------------------------------------------------------------------------------------------------------------------

            . /psdb/PT8.49/psconfigNOMPRREG.sh

             

            echo Inicia el procedimiento para Monitorear Servicios de Tuxedo >  /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

            echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

            echo Application Server NOMPRREG >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

            echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

             

            export TUXCONFIG=/psdb/PT8.49/appserv/NOMPRREG/PSTUXCFG

            /psdb/bea/tuxedo91/bin/tmadmin < /psdb/utilerias/monitoreotuxedo/comandostmadmin.in >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

             

            echo >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

            echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

            echo Process Scheduler NOMPRREG >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

            echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

            export TUXCONFIG=/psdb/PT8.49/appserv/prcs/NOMPRREG/PSTUXCFG

            /psdb/bea/tuxedo91/bin/tmadmin < /psdb/utilerias/monitoreotuxedo/comandostmadmin.in >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

             

            # Begin the code for send email the file /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

            #...

            -------------------------------------------------------

             

            And the file comandostmadmin.in

            -------------------------------------------------------

            psr

            psc

            pclt

            pq

            q

            -------------------------------------------------------

             

             

             

            From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of David Kurtz
            Sent: viernes, 01 de marzo de 2013 11:52 a.m.
            To: psftdba@yahoogroups.com
            Subject: RE: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

             

             

            By coincidence, one of my customers has been experiencing the same problem.  It is not completely clear what is the exact cause.  There is some suggestion that LIBTUX_CAT:577 is a symptom of a corruption in the bulletin board but I have no direct evidence for this.   

            In our case we were running Tuxedo 9.1 with the base rolling patch - level 036.  Oracle support recommended applying rolling patch 095 - and there are various support notes that support this recommendation.  We have done so and are now waiting for the next peak processing day when we have had the errors.

            I don't completely agree with note 764563.1 - only one instance of the restartsrv can exist in a single domain, so there is never concurrent recycling.  Recycling of one server can be deferred while another is recycled.  However, very frequent recycling in a busy domain may be able produce these conditions.

            Psadmin takes a lock out on the bulletin board.  If you are writing scripts that use tmadmin to interrogate the BB and collect metrics then use tmadmin -r because it doesn't lock the BB in administrative mode.  If you open tmadin with the -r option twice you may see.

            TMADMIN_CAT:199: WARN: Cannot become administrator.Limited set of commands available.

            I can also reproduce the LIBTUX_CAT:577 error during recycling, but this is generated by the tmadmin process itself, rather than by BBL process as we are experiencing.

            230809.GO-FASTER-6!tmadmin.6652.2800.-2: LIBTUX_CAT:577: ERROR: Unable to register because the slot is already owned by another process

            230821.GO-FASTER-6!tmadmin.7084.7852.-2: 02-26-2013: Tuxedo Version 10.3.0.0 with VS2008, 32-bit

            230821.GO-FASTER-6!tmadmin.7084.7852.-2: TMADMIN_CAT:1330: INFO: Command: stop -i 2

            231013.GO-FASTER-6!PSAPPSRV.6528.5656.0: LIBTUX_CAT:1393: WARN: tpreturn called with TPEXIT

            231014.GO-FASTER-6!JSH.7200.7204.-2: JOLT_CAT:1043: "ERROR: tpacall() call failed, tperrno = 6"

            231014.GO-FASTER-6!JSH.7200.7204.-2: JOLT_CAT:1043: "ERROR: tpacall() call failed, tperrno = 6"

            regards

            _________________________

            David Kurtz

            tel: +44 (0)7771 760660

            mailto:david.kurtz@...

             

            -----Original Message-----
            From:
            psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of tdcornwell_120maple
            Sent: 26 February 2013 22:20
            To:
            psftdba@yahoogroups.com
            Subject: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

            I have noticed that running a command line invocation of tmadmin on a running domain will *sometimes* generate an error:

            "LIBTUX_CAT:577: Unable to register because the slot is already owned by another process"  I can get this by running something as simple as:

             "psadmin -c [sstatus, or cstatus, or qstatus] -d mydomain"

            I have tracked this down to an ORA doc [ID 764563.1] but the description isn't very encouraging:

            "...Such an error is reported by a TUXEDO administrative process that is trying to register in the Bulletin Board. For recovery purposes, some TUXEDO administrative processes (restartsrv, cleanupsrv, ...) use a reserved slot in the BB.

            So running multiple instances of the same process may cause the first instance to register using the reserved slot and subsequent instances to report LIBTUX_CAT:577: ERROR: Unable to register because the slot is already owned by another process...."

            The reason I'm concerned is because, if I run some of these command line invocations to gather monitoring information on the domain, and wonder if this actually can cause the running instance to fail?

            I know that on occasion, starting the domain fails, and I'm pretty sure that the monitoring task is the cause.

            Is there a way to avoid this?

            Thanks,

            -T

             

             




          Your message has been successfully submitted and would be delivered to recipients shortly.