Loading ...
Sorry, an error occurred while loading the content.

RE: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

Expand Messages
  • David Kurtz
    By coincidence, one of my customers has been experiencing the same problem. It is not completely clear what is the exact cause. There is some suggestion that
    Message 1 of 4 , Mar 1, 2013
    • 0 Attachment
      RE: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

      By coincidence, one of my customers has been experiencing the same problem.  It is not completely clear what is the exact cause.  There is some suggestion that LIBTUX_CAT:577 is a symptom of a corruption in the bulletin board but I have no direct evidence for this.   

      In our case we were running Tuxedo 9.1 with the base rolling patch - level 036.  Oracle support recommended applying rolling patch 095 - and there are various support notes that support this recommendation.  We have done so and are now waiting for the next peak processing day when we have had the errors.

      I don't completely agree with note 764563.1 - only one instance of the restartsrv can exist in a single domain, so there is never concurrent recycling.  Recycling of one server can be deferred while another is recycled.  However, very frequent recycling in a busy domain may be able produce these conditions.

      Psadmin takes a lock out on the bulletin board.  If you are writing scripts that use tmadmin to interrogate the BB and collect metrics then use tmadmin -r because it doesn't lock the BB in administrative mode.  If you open tmadin with the -r option twice you may see.

      TMADMIN_CAT:199: WARN: Cannot become administrator.Limited set of commands available.

      I can also reproduce the LIBTUX_CAT:577 error during recycling, but this is generated by the tmadmin process itself, rather than by BBL process as we are experiencing.

      230809.GO-FASTER-6!tmadmin.6652.2800.-2: LIBTUX_CAT:577: ERROR: Unable to register because the slot is already owned by another process

      230821.GO-FASTER-6!tmadmin.7084.7852.-2: 02-26-2013: Tuxedo Version 10.3.0.0 with VS2008, 32-bit

      230821.GO-FASTER-6!tmadmin.7084.7852.-2: TMADMIN_CAT:1330: INFO: Command: stop -i 2

      231013.GO-FASTER-6!PSAPPSRV.6528.5656.0: LIBTUX_CAT:1393: WARN: tpreturn called with TPEXIT

      231014.GO-FASTER-6!JSH.7200.7204.-2: JOLT_CAT:1043: "ERROR: tpacall() call failed, tperrno = 6"

      231014.GO-FASTER-6!JSH.7200.7204.-2: JOLT_CAT:1043: "ERROR: tpacall() call failed, tperrno = 6"

      regards

      _________________________

      David Kurtz

      tel: +44 (0)7771 760660

      mailto:david.kurtz@...



      -----Original Message-----
      From:
      psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of tdcornwell_120maple
      Sent: 26 February 2013 22:20
      To:
      psftdba@yahoogroups.com
      Subject: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

      I have noticed that running a command line invocation of tmadmin on a running domain will *sometimes* generate an error:

      "LIBTUX_CAT:577: Unable to register because the slot is already owned by another process"  I can get this by running something as simple as:

       "psadmin -c [sstatus, or cstatus, or qstatus] -d mydomain"

      I have tracked this down to an ORA doc [ID 764563.1] but the description isn't very encouraging:

      "...Such an error is reported by a TUXEDO administrative process that is trying to register in the Bulletin Board. For recovery purposes, some TUXEDO administrative processes (restartsrv, cleanupsrv, ...) use a reserved slot in the BB.

      So running multiple instances of the same process may cause the first instance to register using the reserved slot and subsequent instances to report LIBTUX_CAT:577: ERROR: Unable to register because the slot is already owned by another process...."

      The reason I'm concerned is because, if I run some of these command line invocations to gather monitoring information on the domain, and wonder if this actually can cause the running instance to fail?

      I know that on occasion, starting the domain fails, and I'm pretty sure that the monitoring task is the cause.

      Is there a way to avoid this?

      Thanks,

      -T




    • Juarez, Jose Antonio
      Sometime ago I have a similar problem, and I upgraded to the same Rolling Patch that say David Kurtz. Since then I begin to use tmadmin instead of psadmin to
      Message 2 of 4 , Mar 1, 2013
      • 0 Attachment

        Sometime ago I have a similar problem, and I upgraded to the same Rolling Patch that say David Kurtz.

        Since then I begin  to use tmadmin instead of psadmin to monitor Application Server and Process Scheduler.

         

        This is my script that I have to monitoring services in a cron.

        -------------------------------------------------------

        #!/bin/ksh

        # JAJUPA For monitoring Tuxedo services.

         

        export TUXOFFSET=0

        FechaMon=`date "+%y%m%d"`

        FechaHoy=`date "+%y/%m/%d %H:%M:%S"`

         

        # --------------------------------------------------------------------------------------------------------------------------------

        # Inicia el procedimiento para Monitorear el NOMPRREG -

        # --------------------------------------------------------------------------------------------------------------------------------

        . /psdb/PT8.49/psconfigNOMPRREG.sh

         

        echo Inicia el procedimiento para Monitorear Servicios de Tuxedo >  /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

        echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

        echo Application Server NOMPRREG >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

        echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

         

        export TUXCONFIG=/psdb/PT8.49/appserv/NOMPRREG/PSTUXCFG

        /psdb/bea/tuxedo91/bin/tmadmin < /psdb/utilerias/monitoreotuxedo/comandostmadmin.in >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

         

        echo >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

        echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

        echo Process Scheduler NOMPRREG >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

        echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

        export TUXCONFIG=/psdb/PT8.49/appserv/prcs/NOMPRREG/PSTUXCFG

        /psdb/bea/tuxedo91/bin/tmadmin < /psdb/utilerias/monitoreotuxedo/comandostmadmin.in >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

         

        # Begin the code for send email the file /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

        #...

        -------------------------------------------------------

         

        And the file comandostmadmin.in

        -------------------------------------------------------

        psr

        psc

        pclt

        pq

        q

        -------------------------------------------------------

         

         

         

        From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of David Kurtz
        Sent: viernes, 01 de marzo de 2013 11:52 a.m.
        To: psftdba@yahoogroups.com
        Subject: RE: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

         

         

        By coincidence, one of my customers has been experiencing the same problem.  It is not completely clear what is the exact cause.  There is some suggestion that LIBTUX_CAT:577 is a symptom of a corruption in the bulletin board but I have no direct evidence for this.   

        In our case we were running Tuxedo 9.1 with the base rolling patch - level 036.  Oracle support recommended applying rolling patch 095 - and there are various support notes that support this recommendation.  We have done so and are now waiting for the next peak processing day when we have had the errors.

        I don't completely agree with note 764563.1 - only one instance of the restartsrv can exist in a single domain, so there is never concurrent recycling.  Recycling of one server can be deferred while another is recycled.  However, very frequent recycling in a busy domain may be able produce these conditions.

        Psadmin takes a lock out on the bulletin board.  If you are writing scripts that use tmadmin to interrogate the BB and collect metrics then use tmadmin -r because it doesn't lock the BB in administrative mode.  If you open tmadin with the -r option twice you may see.

        TMADMIN_CAT:199: WARN: Cannot become administrator.Limited set of commands available.

        I can also reproduce the LIBTUX_CAT:577 error during recycling, but this is generated by the tmadmin process itself, rather than by BBL process as we are experiencing.

        230809.GO-FASTER-6!tmadmin.6652.2800.-2: LIBTUX_CAT:577: ERROR: Unable to register because the slot is already owned by another process

        230821.GO-FASTER-6!tmadmin.7084.7852.-2: 02-26-2013: Tuxedo Version 10.3.0.0 with VS2008, 32-bit

        230821.GO-FASTER-6!tmadmin.7084.7852.-2: TMADMIN_CAT:1330: INFO: Command: stop -i 2

        231013.GO-FASTER-6!PSAPPSRV.6528.5656.0: LIBTUX_CAT:1393: WARN: tpreturn called with TPEXIT

        231014.GO-FASTER-6!JSH.7200.7204.-2: JOLT_CAT:1043: "ERROR: tpacall() call failed, tperrno = 6"

        231014.GO-FASTER-6!JSH.7200.7204.-2: JOLT_CAT:1043: "ERROR: tpacall() call failed, tperrno = 6"

        regards

        _________________________

        David Kurtz

        tel: +44 (0)7771 760660

        mailto:david.kurtz@...

         

        -----Original Message-----
        From:
        psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of tdcornwell_120maple
        Sent: 26 February 2013 22:20
        To:
        psftdba@yahoogroups.com
        Subject: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

        I have noticed that running a command line invocation of tmadmin on a running domain will *sometimes* generate an error:

        "LIBTUX_CAT:577: Unable to register because the slot is already owned by another process"  I can get this by running something as simple as:

         "psadmin -c [sstatus, or cstatus, or qstatus] -d mydomain"

        I have tracked this down to an ORA doc [ID 764563.1] but the description isn't very encouraging:

        "...Such an error is reported by a TUXEDO administrative process that is trying to register in the Bulletin Board. For recovery purposes, some TUXEDO administrative processes (restartsrv, cleanupsrv, ...) use a reserved slot in the BB.

        So running multiple instances of the same process may cause the first instance to register using the reserved slot and subsequent instances to report LIBTUX_CAT:577: ERROR: Unable to register because the slot is already owned by another process...."

        The reason I'm concerned is because, if I run some of these command line invocations to gather monitoring information on the domain, and wonder if this actually can cause the running instance to fail?

        I know that on occasion, starting the domain fails, and I'm pretty sure that the monitoring task is the cause.

        Is there a way to avoid this?

        Thanks,

        -T



      • David Kurtz
        If you are going to do that, invoke tmadmin with the -r option. As, example, I do in my tuxmon scripts (http://www.go-faster.co.uk/scripts/tuxmon.zip) Thus .
        Message 3 of 4 , Mar 1, 2013
        • 0 Attachment

          If you are going to do that, invoke tmadmin with the –r option.  As, example, I do in my tuxmon scripts (http://www.go-faster.co.uk/scripts/tuxmon.zip)

          Thus

           

          export TUXCONFIG=/psdb/PT8.49/appserv/NOMPRREG/PSTUXCFG

          /psdb/bea/tuxedo91/bin/tmadmin –r < /psdb/utilerias/monitoreotuxedo/comandostmadmin.in >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

           

          echo >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo Process Scheduler NOMPRREG >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          export TUXCONFIG=/psdb/PT8.49/appserv/prcs/NOMPRREG/PSTUXCFG

          /psdb/bea/tuxedo91/bin/tmadmin –r < /psdb/utilerias/monitoreotuxedo/comandostmadmin.in >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

           

          regards
          _________________________
          David Kurtz
          tel: +44 (0)7771 760660
          mailto:david.kurtz@...

           

           

          From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of Juarez, Jose Antonio
          Sent: 01 March 2013 19:07
          To: psftdba@yahoogroups.com
          Subject: RE: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

           




          Sometime ago I have a similar problem, and I upgraded to the same Rolling Patch that say David Kurtz.

          Since then I begin  to use tmadmin instead of psadmin to monitor Application Server and Process Scheduler.

           

          This is my script that I have to monitoring services in a cron.

          -------------------------------------------------------

          #!/bin/ksh

          # JAJUPA For monitoring Tuxedo services.

           

          export TUXOFFSET=0

          FechaMon=`date "+%y%m%d"`

          FechaHoy=`date "+%y/%m/%d %H:%M:%S"`

           

          # --------------------------------------------------------------------------------------------------------------------------------

          # Inicia el procedimiento para Monitorear el NOMPRREG -

          # --------------------------------------------------------------------------------------------------------------------------------

          . /psdb/PT8.49/psconfigNOMPRREG.sh

           

          echo Inicia el procedimiento para Monitorear Servicios de Tuxedo >  /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo Application Server NOMPRREG >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

           

          export TUXCONFIG=/psdb/PT8.49/appserv/NOMPRREG/PSTUXCFG

          /psdb/bea/tuxedo91/bin/tmadmin < /psdb/utilerias/monitoreotuxedo/comandostmadmin.in >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

           

          echo >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo Process Scheduler NOMPRREG >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          echo ----------------------------------------------------------- >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          export TUXCONFIG=/psdb/PT8.49/appserv/prcs/NOMPRREG/PSTUXCFG

          /psdb/bea/tuxedo91/bin/tmadmin < /psdb/utilerias/monitoreotuxedo/comandostmadmin.in >> /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

           

          # Begin the code for send email the file /psdb/utilerias/monitoreotuxedo/logs/monitoreotuxedo-$FechaMon.txt

          #...

          -------------------------------------------------------

           

          And the file comandostmadmin.in

          -------------------------------------------------------

          psr

          psc

          pclt

          pq

          q

          -------------------------------------------------------

           

           

           

          From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of David Kurtz
          Sent: viernes, 01 de marzo de 2013 11:52 a.m.
          To: psftdba@yahoogroups.com
          Subject: RE: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

           

           

          By coincidence, one of my customers has been experiencing the same problem.  It is not completely clear what is the exact cause.  There is some suggestion that LIBTUX_CAT:577 is a symptom of a corruption in the bulletin board but I have no direct evidence for this.   

          In our case we were running Tuxedo 9.1 with the base rolling patch - level 036.  Oracle support recommended applying rolling patch 095 - and there are various support notes that support this recommendation.  We have done so and are now waiting for the next peak processing day when we have had the errors.

          I don't completely agree with note 764563.1 - only one instance of the restartsrv can exist in a single domain, so there is never concurrent recycling.  Recycling of one server can be deferred while another is recycled.  However, very frequent recycling in a busy domain may be able produce these conditions.

          Psadmin takes a lock out on the bulletin board.  If you are writing scripts that use tmadmin to interrogate the BB and collect metrics then use tmadmin -r because it doesn't lock the BB in administrative mode.  If you open tmadin with the -r option twice you may see.

          TMADMIN_CAT:199: WARN: Cannot become administrator.Limited set of commands available.

          I can also reproduce the LIBTUX_CAT:577 error during recycling, but this is generated by the tmadmin process itself, rather than by BBL process as we are experiencing.

          230809.GO-FASTER-6!tmadmin.6652.2800.-2: LIBTUX_CAT:577: ERROR: Unable to register because the slot is already owned by another process

          230821.GO-FASTER-6!tmadmin.7084.7852.-2: 02-26-2013: Tuxedo Version 10.3.0.0 with VS2008, 32-bit

          230821.GO-FASTER-6!tmadmin.7084.7852.-2: TMADMIN_CAT:1330: INFO: Command: stop -i 2

          231013.GO-FASTER-6!PSAPPSRV.6528.5656.0: LIBTUX_CAT:1393: WARN: tpreturn called with TPEXIT

          231014.GO-FASTER-6!JSH.7200.7204.-2: JOLT_CAT:1043: "ERROR: tpacall() call failed, tperrno = 6"

          231014.GO-FASTER-6!JSH.7200.7204.-2: JOLT_CAT:1043: "ERROR: tpacall() call failed, tperrno = 6"

          regards

          _________________________

          David Kurtz

          tel: +44 (0)7771 760660

          mailto:david.kurtz@...

           

          -----Original Message-----
          From:
          psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of tdcornwell_120maple
          Sent: 26 February 2013 22:20
          To:
          psftdba@yahoogroups.com
          Subject: PeopleSoft DBA Forum tuxedo error when monitoring an application server domain.

          I have noticed that running a command line invocation of tmadmin on a running domain will *sometimes* generate an error:

          "LIBTUX_CAT:577: Unable to register because the slot is already owned by another process"  I can get this by running something as simple as:

           "psadmin -c [sstatus, or cstatus, or qstatus] -d mydomain"

          I have tracked this down to an ORA doc [ID 764563.1] but the description isn't very encouraging:

          "...Such an error is reported by a TUXEDO administrative process that is trying to register in the Bulletin Board. For recovery purposes, some TUXEDO administrative processes (restartsrv, cleanupsrv, ...) use a reserved slot in the BB.

          So running multiple instances of the same process may cause the first instance to register using the reserved slot and subsequent instances to report LIBTUX_CAT:577: ERROR: Unable to register because the slot is already owned by another process...."

          The reason I'm concerned is because, if I run some of these command line invocations to gather monitoring information on the domain, and wonder if this actually can cause the running instance to fail?

          I know that on occasion, starting the domain fails, and I'm pretty sure that the monitoring task is the cause.

          Is there a way to avoid this?

          Thanks,

          -T

           

           




        Your message has been successfully submitted and would be delivered to recipients shortly.