Loading ...
Sorry, an error occurred while loading the content.

AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

Expand Messages
  • ajay narula
    Hi All,   Recently I have started facing the issue for one of my PS HRMS instances. HCM: 9.1 , PT: 8.52   The PIA hangs and when I check for the Doamin
    Message 1 of 10 , May 14, 2012
    • 0 Attachment
      Hi All,
       
      Recently I have started facing the issue for one of my PS HRMS instances.
      HCM: 9.1 ,
      PT: 8.52
       
      The PIA hangs and when I check for the Doamin Status in psadmin I get as follows - ICPanel for PSAPPSRV Programs.
      I end up bouncing the AppServer and CACHE clear and all works normal for a day or so and the same happens again.
       
      Please could you help let me know any pointers for this. - I have check the tuxedo timeout paramaters as well as the timeout in the Webprofile configuration.
       
       
      > Prog Name      Queue Name  Grp Name      ID RqDone Load Done Current Service
      ---------      ----------  --------      -- ------ --------- ---------------
      BBL            58268       XXXXXXXX       0    366     18300 (  IDLE )
      PSMONITORSRV   MONITOR     MONITOR        1      0         0 (  IDLE )
      PSANALYTICSRV  00080.00001 ANALYTI+       1      0         0 (  IDLE )
      PSAPPSRV       APPQ        APPSRV         1    308     15400 ICPanel
      PSWATCHSRV     WATCH       WATCH          1      0         0 (  IDLE )
      PSANALYTICSRV  00080.00002 ANALYTI+       2      0         0 (  IDLE )
      PSAPPSRV       APPQ        APPSRV         2    616     30800 ICPanel
      PSANALYTICSRV  00080.00003 ANALYTI+       3      0         0 (  IDLE )
      JRAD           00095.02501 JSLGRP      2501      0         0 (  IDLE )
      PSQRYSRV       QRYQ        APPSRV        70      0         0 (  IDLE )
      PSQRYSRV       QRYQ        APPSRV        71      0         0 (  IDLE )
      JSL            00095.00200 JSLGRP       200      0         0 (  IDLE )
      WSL            00001.00020 BASE          20      0         0 (  IDLE )
      PSMSGDSP       00098.00100 PUBSUB       100     34      1700 (  IDLE )
      PSSAMSRV       SAMQ        APPSRV       100      0         0 (  IDLE )
      PSMSGHND       MBHQ        PUBSUB       101     22      1100 (  IDLE )
      PSRENSRV       RENQ1       RENGRP       101      0         0 (  IDLE )
      PSSAMSRV       SAMQ        APPSRV       101      0         0 (  IDLE )
      PSQCKSRV       QCKQ        APPSRV        50      0         0 (  IDLE )
      PSQCKSRV       QCKQ        APPSRV        51      0         0 (  IDLE )
      JREPSVR        00094.00250 JREPGRP      250      9       450 (  IDLE )
       
       
      Awaiting youe expert suggestions.
       
       
      Thanks
      Ajay
    • David Kurtz
      What this shows is that two PSAPPSRV processes are busy running ICPanel services. If this persists and the RqDone doesn t increase then you have a problem with
      Message 2 of 10 , May 14, 2012
      • 0 Attachment

        What this shows is that two PSAPPSRV processes are busy running ICPanel services. 

        If this persists and the RqDone doesn’t increase then you have a problem with what is going on inside that particular service.

        Increasing the service timeout (the default is 300 seconds) will allow the service to run for longer before Tuxedo intervenes. 

        If Tuxedo is not able to kill the process after the service timeout then that is unusual and needs to be investigated.

        But increasing timeouts is generally the wrong reaction. 

         

        The question needs to be answered is:

        what the service doing that is taking so long?

                        Is it stuck on a piece of SQL or somewhere inside either the component processor or PeopleCode?

         

        From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of ajay narula
        Sent: 14 May 2012 10:40
        To: psftdba@yahoogroups.com
        Subject: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

         




        Hi All,

         

        Recently I have started facing the issue for one of my PS HRMS instances.

        HCM: 9.1 ,

        PT: 8.52

         

        The PIA hangs and when I check for the Doamin Status in psadmin I get as follows - ICPanel for PSAPPSRV Programs.

        I end up bouncing the AppServer and CACHE clear and all works normal for a day or so and the same happens again.

         

        Please could you help let me know any pointers for this. - I have check the tuxedo timeout paramaters as well as the timeout in the Webprofile configuration.

         

         

        > Prog Name      Queue Name  Grp Name      ID RqDone Load Done Current Service
        ---------      ----------  --------      -- ------ --------- ---------------
        BBL            58268       XXXXXXXX       0    366     18300 (  IDLE )
        PSMONITORSRV   MONITOR     MONITOR        1      0         0 (  IDLE )
        PSANALYTICSRV  00080.00001 ANALYTI+       1      0         0 (  IDLE )
        PSAPPSRV       APPQ        APPSRV         1    308     15400 ICPanel
        PSWATCHSRV     WATCH       WATCH          1      0         0 (  IDLE )
        PSANALYTICSRV  00080.00002 ANALYTI+       2      0         0 (  IDLE )
        PSAPPSRV       APPQ        APPSRV         2    616     30800 ICPanel
        PSANALYTICSRV  00080.00003 ANALYTI+       3      0         0 (  IDLE )
        JRAD           00095.02501 JSLGRP      2501      0         0 (  IDLE )
        PSQRYSRV       QRYQ        APPSRV        70      0         0 (  IDLE )
        PSQRYSRV       QRYQ        APPSRV        71      0         0 (  IDLE )
        JSL            00095.00200 JSLGRP       200      0         0 (  IDLE )
        WSL            00001.00020 BASE          20      0         0 (  IDLE )
        PSMSGDSP       00098.00100 PUBSUB       100     34      1700 (  IDLE )
        PSSAMSRV       SAMQ        APPSRV       100      0         0 (  IDLE )
        PSMSGHND       MBHQ        PUBSUB       101     22      1100 (  IDLE )
        PSRENSRV       RENQ1       RENGRP       101      0         0 (  IDLE )
        PSSAMSRV       SAMQ        APPSRV       101      0         0 (  IDLE )
        PSQCKSRV       QCKQ        APPSRV        50      0         0 (  IDLE )
        PSQCKSRV       QCKQ        APPSRV        51      0         0 (  IDLE )
        JREPSVR        00094.00250 JREPGRP      250      9       450 (  IDLE )

         

         

        Awaiting youe expert suggestions.

         

         

        Thanks

        Ajay




      • the_dragon Draco
        In addition to what David said, this usually points to a custom component with crap sql code behind it. Run a trace and maybe you can catch it. Your users
        Message 3 of 10 , May 14, 2012
        • 0 Attachment
          In addition to what David said, this usually points to a custom component with crap sql code behind it.
           
          Run a trace and maybe you can catch it.  Your users should be able to help you determine what is taking so long because they should be whining loud and clear.

          peace,
          clark 'the dragon' willis

           

          To: psftdba@yahoogroups.com
          From: david.kurtz@...
          Date: Mon, 14 May 2012 10:56:54 +0100
          Subject: RE: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

           

          What this shows is that two PSAPPSRV processes are busy running ICPanel services. 

          If this persists and the RqDone doesn’t increase then you have a problem with what is going on inside that particular service.

          Increasing the service timeout (the default is 300 seconds) will allow the service to run for longer before Tuxedo intervenes. 

          If Tuxedo is not able to kill the process after the service timeout then that is unusual and needs to be investigated.

          But increasing timeouts is generally the wrong reaction. 

           

          The question needs to be answered is:

          what the service doing that is taking so long?

                          Is it stuck on a piece of SQL or somewhere inside either the component processor or PeopleCode?

           

          From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of ajay narula
          Sent: 14 May 2012 10:40
          To: psftdba@yahoogroups.com
          Subject: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

           




          Hi All,

           

          Recently I have started facing the issue for one of my PS HRMS instances.

          HCM: 9.1 ,

          PT: 8.52

           

          The PIA hangs and when I check for the Doamin Status in psadmin I get as follows - ICPanel for PSAPPSRV Programs.

          I end up bouncing the AppServer and CACHE clear and all works normal for a day or so and the same happens again.

           

          Please could you help let me know any pointers for this. - I have check the tuxedo timeout paramaters as well as the timeout in the Webprofile configuration.

           

           

          > Prog Name      Queue Name  Grp Name      ID RqDone Load Done Current Service

          ---------      ----------  --------      -- ------ --------- ---------------
          BBL            58268       XXXXXXXX       0    366     18300 (  IDLE )
          PSMONITORSRV   MONITOR     MONITOR        1      0         0 (  IDLE )
          PSANALYTICSRV  00080.00001 ANALYTI+       1      0         0 (  IDLE )
          PSAPPSRV       APPQ        APPSRV         1    308     15400 ICPanel
          PSWATCHSRV     WATCH       WATCH          1      0         0 (  IDLE )
          PSANALYTICSRV  00080.00002 ANALYTI+       2      0         0 (  IDLE )
          PSAPPSRV       APPQ        APPSRV         2    616     30800 ICPanel
          PSANALYTICSRV  00080.00003 ANALYTI+       3      0         0 (  IDLE )
          JRAD           00095.02501 JSLGRP      2501      0         0 (  IDLE )
          PSQRYSRV       QRYQ        APPSRV        70      0         0 (  IDLE )
          PSQRYSRV       QRYQ        APPSRV        71      0         0 (  IDLE )
          JSL            00095.00200 JSLGRP       200      0         0 (  IDLE )
          WSL            00001.00020 BASE          20      0         0 (  IDLE )
          PSMSGDSP       00098.00100 PUBSUB       100     34      1700 (  IDLE )
          PSSAMSRV       SAMQ        APPSRV       100      0         0 (  IDLE )
          PSMSGHND       MBHQ        PUBSUB       101     22      1100 (  IDLE )
          PSRENSRV       RENQ1       RENGRP       101      0         0 (  IDLE )
          PSSAMSRV       SAMQ        APPSRV       101      0         0 (  IDLE )
          PSQCKSRV       QCKQ        APPSRV        50      0         0 (  IDLE )
          PSQCKSRV       QCKQ        APPSRV        51      0         0 (  IDLE )
          JREPSVR        00094.00250 JREPGRP      250      9       450 (  IDLE )

           

           

          Awaiting youe expert suggestions.

           

           

          Thanks

          Ajay





        • Wong, Jane S.
          Hello, I would really appreciate it someone could take me off this email list. Thank you. Jane Wong Creatives Services Advisor Customer Service &
          Message 4 of 10 , May 14, 2012
          • 0 Attachment

            Hello,

             

            I would really appreciate it someone could take me off this email list.

             

            Thank you.

             

            Jane Wong

            Creatives Services Advisor
            Customer Service & Communications

             

            From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of David Kurtz
            Sent: 2012 May 14 3:57 AM
            To: psftdba@yahoogroups.com
            Subject: RE: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

             

             

            What this shows is that two PSAPPSRV processes are busy running ICPanel services. 

            If this persists and the RqDone doesn’t increase then you have a problem with what is going on inside that particular service.

            Increasing the service timeout (the default is 300 seconds) will allow the service to run for longer before Tuxedo intervenes. 

            If Tuxedo is not able to kill the process after the service timeout then that is unusual and needs to be investigated.

            But increasing timeouts is generally the wrong reaction. 

             

            The question needs to be answered is:

            what the service doing that is taking so long?

                            Is it stuck on a piece of SQL or somewhere inside either the component processor or PeopleCode?

             

            From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of ajay narula
            Sent: 14 May 2012 10:40
            To: psftdba@yahoogroups.com
            Subject: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

             





            Hi All,

             

            Recently I have started facing the issue for one of my PS HRMS instances.

            HCM: 9.1 ,

            PT: 8.52

             

            The PIA hangs and when I check for the Doamin Status in psadmin I get as follows - ICPanel for PSAPPSRV Programs.

            I end up bouncing the AppServer and CACHE clear and all works normal for a day or so and the same happens again.

             

            Please could you help let me know any pointers for this. - I have check the tuxedo timeout paramaters as well as the timeout in the Webprofile configuration.

             

             

            > Prog Name      Queue Name  Grp Name      ID RqDone Load Done Current Service
            ---------      ----------  --------      -- ------ --------- ---------------
            BBL            58268       XXXXXXXX       0    366     18300 (  IDLE )
            PSMONITORSRV   MONITOR     MONITOR        1      0         0 (  IDLE )
            PSANALYTICSRV  00080.00001 ANALYTI+       1      0         0 (  IDLE )
            PSAPPSRV       APPQ        APPSRV         1    308     15400 ICPanel
            PSWATCHSRV     WATCH       WATCH          1      0         0 (  IDLE )
            PSANALYTICSRV  00080.00002 ANALYTI+       2      0         0 (  IDLE )
            PSAPPSRV       APPQ        APPSRV         2    616     30800 ICPanel
            PSANALYTICSRV  00080.00003 ANALYTI+       3      0         0 (  IDLE )
            JRAD           00095.02501 JSLGRP      2501      0         0 (  IDLE )
            PSQRYSRV       QRYQ        APPSRV        70      0         0 (  IDLE )
            PSQRYSRV       QRYQ        APPSRV        71      0         0 (  IDLE )
            JSL            00095.00200 JSLGRP       200      0         0 (  IDLE )
            WSL            00001.00020 BASE          20      0         0 (  IDLE )
            PSMSGDSP       00098.00100 PUBSUB       100     34      1700 (  IDLE )
            PSSAMSRV       SAMQ        APPSRV       100      0         0 (  IDLE )
            PSMSGHND       MBHQ        PUBSUB       101     22      1100 (  IDLE )
            PSRENSRV       RENQ1       RENGRP       101      0         0 (  IDLE )
            PSSAMSRV       SAMQ        APPSRV       101      0         0 (  IDLE )
            PSQCKSRV       QCKQ        APPSRV        50      0         0 (  IDLE )
            PSQCKSRV       QCKQ        APPSRV        51      0         0 (  IDLE )
            JREPSVR        00094.00250 JREPGRP      250      9       450 (  IDLE )

             

             

            Awaiting youe expert suggestions.

             

             

            Thanks

            Ajay







            NOTICE -
            This communication is intended ONLY for the use of the person or entity named above and may contain information that is confidential or legally privileged. If you are not the intended recipient named above or a person responsible for delivering messages or communications to the intended recipient, YOU ARE HEREBY NOTIFIED that any use, distribution, or copying of this communication or any of the information contained in it is strictly prohibited. If you have received this communication in error, please notify us immediately by telephone and then destroy or delete this communication, or return it to us by mail if requested by us. The City of Calgary thanks you for your attention and co-operation.
          • Andy Wheatcroft
            Jane, I believe all you have to do is send an email to: psftdba-unsubscribe@yahoogroups.com and you will be taken off the list. Andy. From: Wong, Jane S.
            Message 5 of 10 , May 14, 2012
            • 0 Attachment
              Jane,

              I believe all you have to do is send an email to:

              psftdba-unsubscribe@yahoogroups.com

              and you will be taken off the list.

              Andy.

              From: "Wong, Jane S." <jane.wong@...>
              Reply-To: <psftdba@yahoogroups.com>
              Date: Mon, 14 May 2012 08:20:38 -0600
              To: "'psftdba@yahoogroups.com'" <psftdba@yahoogroups.com>
              Subject: RE: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

               

              Hello,

               

              I would really appreciate it someone could take me off this email list.

               

              Thank you.

               

              Jane Wong

              Creatives Services Advisor
              Customer Service & Communications

               

              From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of David Kurtz
              Sent: 2012 May 14 3:57 AM
              To: psftdba@yahoogroups.com
              Subject: RE: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

               

               

              What this shows is that two PSAPPSRV processes are busy running ICPanel services. 

              If this persists and the RqDone doesn’t increase then you have a problem with what is going on inside that particular service.

              Increasing the service timeout (the default is 300 seconds) will allow the service to run for longer before Tuxedo intervenes. 

              If Tuxedo is not able to kill the process after the service timeout then that is unusual and needs to be investigated.

              But increasing timeouts is generally the wrong reaction. 

               

              The question needs to be answered is:

              what the service doing that is taking so long?

                              Is it stuck on a piece of SQL or somewhere inside either the component processor or PeopleCode?

               

              From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of ajay narula
              Sent: 14 May 2012 10:40
              To: psftdba@yahoogroups.com
              Subject: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

               





              Hi All,

               

              Recently I have started facing the issue for one of my PS HRMS instances.

              HCM: 9.1 ,

              PT: 8.52

               

              The PIA hangs and when I check for the Doamin Status in psadmin I get as follows - ICPanel for PSAPPSRV Programs.

              I end up bouncing the AppServer and CACHE clear and all works normal for a day or so and the same happens again.

               

              Please could you help let me know any pointers for this. - I have check the tuxedo timeout paramaters as well as the timeout in the Webprofile configuration.

               

               

              > Prog Name      Queue Name  Grp Name      ID RqDone Load Done Current Service

              ---------      ----------  --------      -- ------ --------- ---------------
              BBL            58268       XXXXXXXX       0    366     18300 (  IDLE )
              PSMONITORSRV   MONITOR     MONITOR        1      0         0 (  IDLE )
              PSANALYTICSRV  00080.00001 ANALYTI+       1      0         0 (  IDLE )
              PSAPPSRV       APPQ        APPSRV         1    308     15400 ICPanel
              PSWATCHSRV     WATCH       WATCH          1      0         0 (  IDLE )
              PSANALYTICSRV  00080.00002 ANALYTI+       2      0         0 (  IDLE )
              PSAPPSRV       APPQ        APPSRV         2    616     30800 ICPanel
              PSANALYTICSRV  00080.00003 ANALYTI+       3      0         0 (  IDLE )
              JRAD           00095.02501 JSLGRP      2501      0         0 (  IDLE )
              PSQRYSRV       QRYQ        APPSRV        70      0         0 (  IDLE )
              PSQRYSRV       QRYQ        APPSRV        71      0         0 (  IDLE )
              JSL            00095.00200 JSLGRP       200      0         0 (  IDLE )
              WSL            00001.00020 BASE          20      0         0 (  IDLE )
              PSMSGDSP       00098.00100 PUBSUB       100     34      1700 (  IDLE )
              PSSAMSRV       SAMQ        APPSRV       100      0         0 (  IDLE )
              PSMSGHND       MBHQ        PUBSUB       101     22      1100 (  IDLE )
              PSRENSRV       RENQ1       RENGRP       101      0         0 (  IDLE )
              PSSAMSRV       SAMQ        APPSRV       101      0         0 (  IDLE )
              PSQCKSRV       QCKQ        APPSRV        50      0         0 (  IDLE )
              PSQCKSRV       QCKQ        APPSRV        51      0         0 (  IDLE )
              JREPSVR        00094.00250 JREPGRP      250      9       450 (  IDLE )

               

               

              Awaiting youe expert suggestions.

               

               

              Thanks

              Ajay







              NOTICE -
              This communication is intended ONLY for the use of the person or entity named above and may contain information that is confidential or legally privileged. If you are not the intended recipient named above or a person responsible for delivering messages or communications to the intended recipient, YOU ARE HEREBY NOTIFIED that any use, distribution, or copying of this communication or any of the information contained in it is strictly prohibited. If you have received this communication in error, please notify us immediately by telephone and then destroy or delete this communication, or return it to us by mail if requested by us. The City of Calgary thanks you for your attention and co-operation.

            • soubhagya upadhyay
              Hi, There is definitely a long running process causing the issue. It would be better if you can involve your DBA to find of the long running sqls. Stay with
              Message 6 of 10 , May 14, 2012
              • 0 Attachment


                Hi,
                There is definitely a long running process causing the issue. It would be better if you can involve your DBA to find of the long running sqls. Stay with see reports (for Oracle database ). Also check with your end users what they have started doing recently ,there might be some year end activity going in which is not part of daily hr operations. If you have PPM configured that would help.

                Sent from Yahoo! Mail on Android



                From: the_dragon Draco <ceprn@...>;
                To: <psftdba@yahoogroups.com>;
                Subject: RE: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)
                Sent: Mon, May 14, 2012 10:15:46 AM

                 

                In addition to what David said, this usually points to a custom component with crap sql code behind it.
                 
                Run a trace and maybe you can catch it.  Your users should be able to help you determine what is taking so long because they should be whining loud and clear.

                peace,
                clark 'the dragon' willis

                 

                To: psftdba@yahoogroups.com
                From: david.kurtz@...
                Date: Mon, 14 May 2012 10:56:54 +0100
                Subject: RE: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

                 

                What this shows is that two PSAPPSRV processes are busy running ICPanel services. 

                If this persists and the RqDone doesn’t increase then you have a problem with what is going on inside that particular service.

                Increasing the service timeout (the default is 300 seconds) will allow the service to run for longer before Tuxedo intervenes. 

                If Tuxedo is not able to kill the process after the service timeout then that is unusual and needs to be investigated.

                But increasing timeouts is generally the wrong reaction. 

                 

                The question needs to be answered is:

                what the service doing that is taking so long?

                                Is it stuck on a piece of SQL or somewhere inside either the component processor or PeopleCode?

                 

                From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of ajay narula
                Sent: 14 May 2012 10:40
                To: psftdba@yahoogroups.com
                Subject: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

                 




                Hi All,

                 

                Recently I have started facing the issue for one of my PS HRMS instances.

                HCM: 9.1 ,

                PT: 8.52

                 

                The PIA hangs and when I check for the Doamin Status in psadmin I get as follows - ICPanel for PSAPPSRV Programs.

                I end up bouncing the AppServer and CACHE clear and all works normal for a day or so and the same happens again.

                 

                Please could you help let me know any pointers for this. - I have check the tuxedo timeout paramaters as well as the timeout in the Webprofile configuration.

                 

                 

                > Prog Name      Queue Name  Grp Name      ID RqDone Load Done Current Service

                ---------      ----------  --------      -- ------ --------- ---------------
                BBL            58268       XXXXXXXX       0    366     18300 (  IDLE )
                PSMONITORSRV   MONITOR     MONITOR        1      0         0 (  IDLE )
                PSANALYTICSRV  00080.00001 ANALYTI+       1      0         0 (  IDLE )
                PSAPPSRV       APPQ        APPSRV         1    308     15400 ICPanel
                PSWATCHSRV     WATCH       WATCH          1      0         0 (  IDLE )
                PSANALYTICSRV  00080.00002 ANALYTI+       2      0         0 (  IDLE )
                PSAPPSRV       APPQ        APPSRV         2    616     30800 ICPanel
                PSANALYTICSRV  00080.00003 ANALYTI+       3      0         0 (  IDLE )
                JRAD           00095.02501 JSLGRP      2501      0         0 (  IDLE )
                PSQRYSRV       QRYQ        APPSRV        70      0         0 (  IDLE )
                PSQRYSRV       QRYQ        APPSRV        71      0         0 (  IDLE )
                JSL            00095.00200 JSLGRP       200      0         0 (  IDLE )
                WSL            00001.00020 BASE          20      0         0 (  IDLE )
                PSMSGDSP       00098.00100 PUBSUB       100     34      1700 (  IDLE )
                PSSAMSRV       SAMQ        APPSRV       100      0         0 (  IDLE )
                PSMSGHND       MBHQ        PUBSUB       101     22      1100 (  IDLE )
                PSRENSRV       RENQ1       RENGRP       101      0         0 (  IDLE )
                PSSAMSRV       SAMQ        APPSRV       101      0         0 (  IDLE )
                PSQCKSRV       QCKQ        APPSRV        50      0         0 (  IDLE )
                PSQCKSRV       QCKQ        APPSRV        51      0         0 (  IDLE )
                JREPSVR        00094.00250 JREPGRP      250      9       450 (  IDLE )

                 

                 

                Awaiting youe expert suggestions.

                 

                 

                Thanks

                Ajay





              • abdulmajid shaikh
                Hi All,   I faced a similar issue with the Appserver getting hung in ICPanel. The problem was I was booting more processes then the CPU alloted for that
                Message 7 of 10 , May 22, 2012
                • 0 Attachment
                  Hi All,
                   
                  I faced a similar issue with the Appserver getting hung in ICPanel.
                  The problem was I was booting more processes then the CPU alloted for that hardware.
                   
                  We did our system hardening like max user processes in AIX to maximum and increased the ulimit as well.
                  However i was facing the entire production still hung up.
                   
                  So i reduced the no of Appserver process to min instance =4 and max instance= 10 as I knew that one particular domain can handle atmost 10 Psappsrv processes.
                   
                  This solved my issue in production which was running havoc over a month. Touchwood my environmnet is stable and the client is happy.
                   
                  Thanks,
                  Abdul.

                  From: David Kurtz <david.kurtz@...>
                  To: psftdba@yahoogroups.com
                  Sent: Monday, 14 May 2012 3:26 PM
                  Subject: RE: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

                   
                  What this shows is that two PSAPPSRV processes are busy running ICPanel services. 
                  If this persists and the RqDone doesn’t increase then you have a problem with what is going on inside that particular service.
                  Increasing the service timeout (the default is 300 seconds) will allow the service to run for longer before Tuxedo intervenes. 
                  If Tuxedo is not able to kill the process after the service timeout then that is unusual and needs to be investigated.
                  But increasing timeouts is generally the wrong reaction. 
                   
                  The question needs to be answered is:
                  what the service doing that is taking so long?
                                  Is it stuck on a piece of SQL or somewhere inside either the component processor or PeopleCode?
                   
                  From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of ajay narula
                  Sent: 14 May 2012 10:40
                  To: psftdba@yahoogroups.com
                  Subject: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)
                   



                  Hi All,
                   
                  Recently I have started facing the issue for one of my PS HRMS instances.
                  HCM: 9.1 ,
                  PT: 8.52
                   
                  The PIA hangs and when I check for the Doamin Status in psadmin I get as follows - ICPanel for PSAPPSRV Programs.
                  I end up bouncing the AppServer and CACHE clear and all works normal for a day or so and the same happens again.
                   
                  Please could you help let me know any pointers for this. - I have check the tuxedo timeout paramaters as well as the timeout in the Webprofile configuration.
                   
                   
                  > Prog Name      Queue Name  Grp Name      ID RqDone Load Done Current Service
                  ---------      ----------  --------      -- ------ --------- ---------------
                  BBL            58268       XXXXXXXX       0    366     18300 (  IDLE )
                  PSMONITORSRV   MONITOR     MONITOR        1      0         0 (  IDLE )
                  PSANALYTICSRV  00080.00001 ANALYTI+       1      0         0 (  IDLE )
                  PSAPPSRV       APPQ        APPSRV         1    308     15400 ICPanel
                  PSWATCHSRV     WATCH       WATCH          1      0         0 (  IDLE )
                  PSANALYTICSRV  00080.00002 ANALYTI+       2      0         0 (  IDLE )
                  PSAPPSRV       APPQ        APPSRV         2    616     30800 ICPanel
                  PSANALYTICSRV  00080.00003 ANALYTI+       3      0         0 (  IDLE )
                  JRAD           00095.02501 JSLGRP      2501      0         0 (  IDLE )
                  PSQRYSRV       QRYQ        APPSRV        70      0         0 (  IDLE )
                  PSQRYSRV       QRYQ        APPSRV        71      0         0 (  IDLE )
                  JSL            00095.00200 JSLGRP       200      0         0 (  IDLE )
                  WSL            00001.00020 BASE          20      0         0 (  IDLE )
                  PSMSGDSP       00098.00100 PUBSUB       100     34      1700 (  IDLE )
                  PSSAMSRV       SAMQ        APPSRV       100      0         0 (  IDLE )
                  PSMSGHND       MBHQ        PUBSUB       101     22      1100 (  IDLE )
                  PSRENSRV       RENQ1       RENGRP       101      0         0 (  IDLE )
                  PSSAMSRV       SAMQ        APPSRV       101      0         0 (  IDLE )
                  PSQCKSRV       QCKQ        APPSRV        50      0         0 (  IDLE )
                  PSQCKSRV       QCKQ        APPSRV        51      0         0 (  IDLE )
                  JREPSVR        00094.00250 JREPGRP      250      9       450 (  IDLE )
                   
                   
                  Awaiting youe expert suggestions.
                   
                   
                  Thanks
                  Ajay





                • abdulmajid shaikh
                  Right. This happens when there is lot of customization done by the developers on some delivered objects. Bad queries. ________________________________ From:
                  Message 8 of 10 , May 22, 2012
                  • 0 Attachment
                    Right. This happens when there is lot of customization done by the developers on some delivered objects.
                    Bad queries.

                    From: the_dragon Draco <ceprn@...>
                    To: psftdba@yahoogroups.com
                    Sent: Monday, 14 May 2012 3:45 PM
                    Subject: RE: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

                     
                    In addition to what David said, this usually points to a custom component with crap sql code behind it.
                     
                    Run a trace and maybe you can catch it.  Your users should be able to help you determine what is taking so long because they should be whining loud and clear.

                    peace,
                    clark 'the dragon' willis

                     
                    To: psftdba@yahoogroups.com
                    From: david.kurtz@...
                    Date: Mon, 14 May 2012 10:56:54 +0100
                    Subject: RE: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

                     
                    What this shows is that two PSAPPSRV processes are busy running ICPanel services. 
                    If this persists and the RqDone doesn’t increase then you have a problem with what is going on inside that particular service.
                    Increasing the service timeout (the default is 300 seconds) will allow the service to run for longer before Tuxedo intervenes. 
                    If Tuxedo is not able to kill the process after the service timeout then that is unusual and needs to be investigated.
                    But increasing timeouts is generally the wrong reaction. 
                     
                    The question needs to be answered is:
                    what the service doing that is taking so long?
                                    Is it stuck on a piece of SQL or somewhere inside either the component processor or PeopleCode?
                     
                    From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of ajay narula
                    Sent: 14 May 2012 10:40
                    To: psftdba@yahoogroups.com
                    Subject: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)
                     



                    Hi All,
                     
                    Recently I have started facing the issue for one of my PS HRMS instances.
                    HCM: 9.1 ,
                    PT: 8.52
                     
                    The PIA hangs and when I check for the Doamin Status in psadmin I get as follows - ICPanel for PSAPPSRV Programs.
                    I end up bouncing the AppServer and CACHE clear and all works normal for a day or so and the same happens again.
                     
                    Please could you help let me know any pointers for this. - I have check the tuxedo timeout paramaters as well as the timeout in the Webprofile configuration.
                     
                     
                    > Prog Name      Queue Name  Grp Name      ID RqDone Load Done Current Service
                    ---------      ----------  --------      -- ------ --------- ---------------
                    BBL            58268       XXXXXXXX       0    366     18300 (  IDLE )
                    PSMONITORSRV   MONITOR     MONITOR        1      0         0 (  IDLE )
                    PSANALYTICSRV  00080.00001 ANALYTI+       1      0         0 (  IDLE )
                    PSAPPSRV       APPQ        APPSRV         1    308     15400 ICPanel
                    PSWATCHSRV     WATCH       WATCH          1      0         0 (  IDLE )
                    PSANALYTICSRV  00080.00002 ANALYTI+       2      0         0 (  IDLE )
                    PSAPPSRV       APPQ        APPSRV         2    616     30800 ICPanel
                    PSANALYTICSRV  00080.00003 ANALYTI+       3      0         0 (  IDLE )
                    JRAD           00095.02501 JSLGRP      2501      0         0 (  IDLE )
                    PSQRYSRV       QRYQ        APPSRV        70      0         0 (  IDLE )
                    PSQRYSRV       QRYQ        APPSRV        71      0         0 (  IDLE )
                    JSL            00095.00200 JSLGRP       200      0         0 (  IDLE )
                    WSL            00001.00020 BASE          20      0         0 (  IDLE )
                    PSMSGDSP       00098.00100 PUBSUB       100     34      1700 (  IDLE )
                    PSSAMSRV       SAMQ        APPSRV       100      0         0 (  IDLE )
                    PSMSGHND       MBHQ        PUBSUB       101     22      1100 (  IDLE )
                    PSRENSRV       RENQ1       RENGRP       101      0         0 (  IDLE )
                    PSSAMSRV       SAMQ        APPSRV       101      0         0 (  IDLE )
                    PSQCKSRV       QCKQ        APPSRV        50      0         0 (  IDLE )
                    PSQCKSRV       QCKQ        APPSRV        51      0         0 (  IDLE )
                    JREPSVR        00094.00250 JREPGRP      250      9       450 (  IDLE )
                     
                     
                    Awaiting youe expert suggestions.
                     
                     
                    Thanks
                    Ajay






                  • ajay narula
                    Thanks Abdul , David for your value-able inputs - these really helped. We adjusted the PSAPPSRV and also the Peoplecode which was recently modified had an
                    Message 9 of 10 , Jul 19, 2012
                    • 0 Attachment
                      Thanks Abdul , David for your value-able inputs - these really helped.


                      We adjusted the PSAPPSRV and also the Peoplecode which was recently modified had an issue.

                      All is working fine now.

                      Thanks
                      Ajay


                      From: abdulmajid shaikh <shak_abdul@...>
                      To: "psftdba@yahoogroups.com" <psftdba@yahoogroups.com>
                      Sent: Wednesday, May 23, 2012 12:00 AM
                      Subject: Re: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

                       
                      Hi All,
                       
                      I faced a similar issue with the Appserver getting hung in ICPanel.
                      The problem was I was booting more processes then the CPU alloted for that hardware.
                       
                      We did our system hardening like max user processes in AIX to maximum and increased the ulimit as well.
                      However i was facing the entire production still hung up.
                       
                      So i reduced the no of Appserver process to min instance =4 and max instance= 10 as I knew that one particular domain can handle atmost 10 Psappsrv processes.
                       
                      This solved my issue in production which was running havoc over a month. Touchwood my environmnet is stable and the client is happy.
                       
                      Thanks,
                      Abdul.

                      From: David Kurtz <david.kurtz@...>
                      To: psftdba@yahoogroups.com
                      Sent: Monday, 14 May 2012 3:26 PM
                      Subject: RE: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

                       
                      What this shows is that two PSAPPSRV processes are busy running ICPanel services. 
                      If this persists and the RqDone doesn’t increase then you have a problem with what is going on inside that particular service.
                      Increasing the service timeout (the default is 300 seconds) will allow the service to run for longer before Tuxedo intervenes. 
                      If Tuxedo is not able to kill the process after the service timeout then that is unusual and needs to be investigated.
                      But increasing timeouts is generally the wrong reaction. 
                       
                      The question needs to be answered is:
                      what the service doing that is taking so long?
                                      Is it stuck on a piece of SQL or somewhere inside either the component processor or PeopleCode?
                       
                      From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of ajay narula
                      Sent: 14 May 2012 10:40
                      To: psftdba@yahoogroups.com
                      Subject: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)
                       



                      Hi All,
                       
                      Recently I have started facing the issue for one of my PS HRMS instances.
                      HCM: 9.1 ,
                      PT: 8.52
                       
                      The PIA hangs and when I check for the Doamin Status in psadmin I get as follows - ICPanel for PSAPPSRV Programs.
                      I end up bouncing the AppServer and CACHE clear and all works normal for a day or so and the same happens again.
                       
                      Please could you help let me know any pointers for this. - I have check the tuxedo timeout paramaters as well as the timeout in the Webprofile configuration.
                       
                       
                      > Prog Name      Queue Name  Grp Name      ID RqDone Load Done Current Service
                      ---------      ----------  --------      -- ------ --------- ---------------
                      BBL            58268       XXXXXXXX       0    366     18300 (  IDLE )
                      PSMONITORSRV   MONITOR     MONITOR        1      0         0 (  IDLE )
                      PSANALYTICSRV  00080.00001 ANALYTI+       1      0         0 (  IDLE )
                      PSAPPSRV       APPQ        APPSRV         1    308     15400 ICPanel
                      PSWATCHSRV     WATCH       WATCH          1      0         0 (  IDLE )
                      PSANALYTICSRV  00080.00002 ANALYTI+       2      0         0 (  IDLE )
                      PSAPPSRV       APPQ        APPSRV         2    616     30800 ICPanel
                      PSANALYTICSRV  00080.00003 ANALYTI+       3      0         0 (  IDLE )
                      JRAD           00095.02501 JSLGRP      2501      0         0 (  IDLE )
                      PSQRYSRV       QRYQ        APPSRV        70      0         0 (  IDLE )
                      PSQRYSRV       QRYQ        APPSRV        71      0         0 (  IDLE )
                      JSL            00095.00200 JSLGRP       200      0         0 (  IDLE )
                      WSL            00001.00020 BASE          20      0         0 (  IDLE )
                      PSMSGDSP       00098.00100 PUBSUB       100     34      1700 (  IDLE )
                      PSSAMSRV       SAMQ        APPSRV       100      0         0 (  IDLE )
                      PSMSGHND       MBHQ        PUBSUB       101     22      1100 (  IDLE )
                      PSRENSRV       RENQ1       RENGRP       101      0         0 (  IDLE )
                      PSSAMSRV       SAMQ        APPSRV       101      0         0 (  IDLE )
                      PSQCKSRV       QCKQ        APPSRV        50      0         0 (  IDLE )
                      PSQCKSRV       QCKQ        APPSRV        51      0         0 (  IDLE )
                      JREPSVR        00094.00250 JREPGRP      250      9       450 (  IDLE )
                       
                       
                      Awaiting youe expert suggestions.
                       
                       
                      Thanks
                      Ajay







                    • Nalla Sivam
                      It has been a while, we faced these issue and also it was on older tools version like 8.45+ , 8.49+ . We came across this and there was known issues with
                      Message 10 of 10 , Jul 20, 2012
                      • 0 Attachment
                        It has been a while, we faced these issue and also it was on older tools version like 8.45+ , 8.49+ .

                        We came across this and there was known issues with spawning, did you try keeping max and min instance same instead "instance =4 and max instance= 10".

                        Do timeout values are set such that the values are greater as we move away from db  (db timeout < app svr timeout < wl timeout ), so same request do not keep coming from web to db.  ( note: this may be reveres - check it out)
                         



                        thanks
                        Nalla


                        From: ajay narula <narulajy@...>
                        To: "psftdba@yahoogroups.com" <psftdba@yahoogroups.com>
                        Cc: "shak_abdul@..." <shak_abdul@...>; "david.kurtz@..." <david.kurtz@...>
                        Sent: Thursday, July 19, 2012 8:17 PM
                        Subject: Re: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

                         
                        Thanks Abdul , David for your value-able inputs - these really helped.


                        We adjusted the PSAPPSRV and also the Peoplecode which was recently modified had an issue.

                        All is working fine now.

                        Thanks
                        Ajay


                        From: abdulmajid shaikh <shak_abdul@...>
                        To: "psftdba@yahoogroups.com" <psftdba@yahoogroups.com>
                        Sent: Wednesday, May 23, 2012 12:00 AM
                        Subject: Re: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

                         
                        Hi All,
                         
                        I faced a similar issue with the Appserver getting hung in ICPanel.
                        The problem was I was booting more processes then the CPU alloted for that hardware.
                         
                        We did our system hardening like max user processes in AIX to maximum and increased the ulimit as well.
                        However i was facing the entire production still hung up.
                         
                        So i reduced the no of Appserver process to min instance =4 and max instance= 10 as I knew that one particular domain can handle atmost 10 Psappsrv processes.
                         
                        This solved my issue in production which was running havoc over a month. Touchwood my environmnet is stable and the client is happy.
                         
                        Thanks,
                        Abdul.

                        From: David Kurtz <david.kurtz@...>
                        To: psftdba@yahoogroups.com
                        Sent: Monday, 14 May 2012 3:26 PM
                        Subject: RE: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)

                         
                        What this shows is that two PSAPPSRV processes are busy running ICPanel services. 
                        If this persists and the RqDone doesn’t increase then you have a problem with what is going on inside that particular service.
                        Increasing the service timeout (the default is 300 seconds) will allow the service to run for longer before Tuxedo intervenes. 
                        If Tuxedo is not able to kill the process after the service timeout then that is unusual and needs to be investigated.
                        But increasing timeouts is generally the wrong reaction. 
                         
                        The question needs to be answered is:
                        what the service doing that is taking so long?
                                        Is it stuck on a piece of SQL or somewhere inside either the component processor or PeopleCode?
                         
                        From: psftdba@yahoogroups.com [mailto:psftdba@yahoogroups.com] On Behalf Of ajay narula
                        Sent: 14 May 2012 10:40
                        To: psftdba@yahoogroups.com
                        Subject: PeopleSoft DBA Forum AppServer Hangs PIA shows processing (ICPanel - PSAPPSRV)
                         



                        Hi All,
                         
                        Recently I have started facing the issue for one of my PS HRMS instances.
                        HCM: 9.1 ,
                        PT: 8.52
                         
                        The PIA hangs and when I check for the Doamin Status in psadmin I get as follows - ICPanel for PSAPPSRV Programs.
                        I end up bouncing the AppServer and CACHE clear and all works normal for a day or so and the same happens again.
                         
                        Please could you help let me know any pointers for this. - I have check the tuxedo timeout paramaters as well as the timeout in the Webprofile configuration.
                         
                         
                        > Prog Name      Queue Name  Grp Name      ID RqDone Load Done Current Service
                        ---------      ----------  --------      -- ------ --------- ---------------
                        BBL            58268       XXXXXXXX       0    366     18300 (  IDLE )
                        PSMONITORSRV   MONITOR     MONITOR        1      0         0 (  IDLE )
                        PSANALYTICSRV  00080.00001 ANALYTI+       1      0         0 (  IDLE )
                        PSAPPSRV       APPQ        APPSRV         1    308     15400 ICPanel
                        PSWATCHSRV     WATCH       WATCH          1      0         0 (  IDLE )
                        PSANALYTICSRV  00080.00002 ANALYTI+       2      0         0 (  IDLE )
                        PSAPPSRV       APPQ        APPSRV         2    616     30800 ICPanel
                        PSANALYTICSRV  00080.00003 ANALYTI+       3      0         0 (  IDLE )
                        JRAD           00095.02501 JSLGRP      2501      0         0 (  IDLE )
                        PSQRYSRV       QRYQ        APPSRV        70      0         0 (  IDLE )
                        PSQRYSRV       QRYQ        APPSRV        71      0         0 (  IDLE )
                        JSL            00095.00200 JSLGRP       200      0         0 (  IDLE )
                        WSL            00001.00020 BASE          20      0         0 (  IDLE )
                        PSMSGDSP       00098.00100 PUBSUB       100     34      1700 (  IDLE )
                        PSSAMSRV       SAMQ        APPSRV       100      0         0 (  IDLE )
                        PSMSGHND       MBHQ        PUBSUB       101     22      1100 (  IDLE )
                        PSRENSRV       RENQ1       RENGRP       101      0         0 (  IDLE )
                        PSSAMSRV       SAMQ        APPSRV       101      0         0 (  IDLE )
                        PSQCKSRV       QCKQ        APPSRV        50      0         0 (  IDLE )
                        PSQCKSRV       QCKQ        APPSRV        51      0         0 (  IDLE )
                        JREPSVR        00094.00250 JREPGRP      250      9       450 (  IDLE )
                         
                         
                        Awaiting youe expert suggestions.
                         
                         
                        Thanks
                        Ajay









                      Your message has been successfully submitted and would be delivered to recipients shortly.