Loading ...
Sorry, an error occurred while loading the content.

Re: [itsoracleteam] Global Enqueue Services Deadlock detected

Expand Messages
  • Prashant Hirani
    HItesh, Are you experiencing any cluster interconnect issues ? drops/collisions/timeouts? If not then, you have possibility to disable resource remastering by
    Message 1 of 5 , May 1 6:23 AM
    • 0 Attachment
      HItesh,
       
      Are you experiencing any cluster interconnect issues ? drops/collisions/timeouts?
       
      If not then, you have possibility to disable resource remastering by setting

      _lm_dynamic_remastering=false

      but changing any hidden parameters should be done under advisory from oracle support.
      Also i would advise to upgrade RDBMS/ASM/CRS Version from 10.2.0.3.0 to 10.2.0.4.0 and then apply PSU patch on all the said binaries.
       
      Try to use Metalink note: 398519.1 if you have metalink access.
      -- Prashant Hirani
      On Sat, May 1, 2010 at 9:56 AM, hit_gon <hit_gon@...> wrote:
       

      Hi,

      any one can help me please ?

      I'm frequently detected following error in my alert log file.

      Global Enqueue Services Deadlock detected. More info in file
      /u001/app/oracle/product/10.2.0_db/db_1/admin/XLITEAAA/bdump/Xlitepun1_lmd0_55822.trc.

      DB Version:
      Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bi
      PL/SQL Release 10.2.0.3.0 - Production
      CORE 10.2.0.3.0 Production
      TNS for Solaris: Version 10.2.0.3.0 - Production
      NLSRTL Version 10.2.0.3.0 - Production

      Thanks in Advance.


    • atri prashant
      4913415 Global deadlock not reported in RAC Hi,   I would suggest to go through with the corresponding trace files, what exactly it is showing like Single
      Message 2 of 5 , May 1 11:49 AM
      • 0 Attachment
        4913415Global deadlock not reported in RAC
        Hi,
         
        I would suggest to go through with the corresponding trace files, what exactly it is showing like "Single resource deadlock" or anything else...
         
        When you troubleshoot something, please check associated all supportive error/warning messages and trace files as well then take the final decision.
         
        And if
         
        The corresponding trace file contains the following messages:

              Single resource deadlock: blocking enqueue which blocks itself, f 1

              resname : [0xbb7cc5db][0x82d0d4b5],[IV]

        There are no reported external errors like:

              ORA-60 deadlock detected while waiting for resource
              ORA-4020 "deadlock detected while trying to lock object %s%s%s%s%s"
        These messages can be ignore.

        They do not represent a failure or problem.
        Regards,
        Prashant Atri

        --- On Sat, 1/5/10, Prashant Hirani <hirani.prashant2000@...> wrote:

        From: Prashant Hirani <hirani.prashant2000@...>
        Subject: Re: [itsoracleteam] Global Enqueue Services Deadlock detected
        To: itsoracleteam@yahoogroups.com, hit_gon@...
        Received: Saturday, 1 May, 2010, 6:53 PM

         
        HItesh,
         
        Are you experiencing any cluster interconnect issues ? drops/collisions/ timeouts?
         
        If not then, you have possibility to disable resource remastering by setting

        _lm_dynamic_ remastering= false

        but changing any hidden parameters should be done under advisory from oracle support.
        Also i would advise to upgrade RDBMS/ASM/CRS Version from 10.2.0.3.0 to 10.2.0.4.0 and then apply PSU patch on all the said binaries.
         
        Try to use Metalink note: 398519.1 if you have metalink access.
        -- Prashant Hirani
        On Sat, May 1, 2010 at 9:56 AM, hit_gon <hit_gon@yahoo. com> wrote:
         
        Hi,

        any one can help me please ?

        I'm frequently detected following error in my alert log file.

        Global Enqueue Services Deadlock detected. More info in file
        /u001/app/oracle/ product/10. 2.0_db/db_ 1/admin/XLITEAAA /bdump/Xlitepun1 _lmd0_55822. trc.

        DB Version:
        Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bi
        PL/SQL Release 10.2.0.3.0 - Production
        CORE 10.2.0.3.0 Production
        TNS for Solaris: Version 10.2.0.3.0 - Production
        NLSRTL Version 10.2.0.3.0 - Production

        Thanks in Advance.



         
      • dhrumil patel
        Hi You can consider Global Deadlock in rac database as a normal deadlock as in single instance database. Now in your case try to find the problem area to
        Message 3 of 5 , May 3 11:15 AM
        • 0 Attachment
          Hi

          You can consider Global Deadlock in rac database as a "normal" deadlock as in single instance database. Now in your case try to find the problem area to resolve the issue by analyzing the trace file generated by oracle.

          Check out Global Wait-For_Graph (WFG) from relevant trace file......

          i.e.
          Global Wait-For-Graph( WFG) at ddTS[0.f5] :
          BLOCKED 5f46897b8 5 [0x2090018][ 0x59c9],[ TX] [201C-01CA-00000004 ] 1
          BLOCKER 5f3701920 5 [0x2090018][ 0x59c9],[ TX] [2011-0119-0000000B ] 1
          BLOCKED 5f3701920 5 [0x2090018][ 0x59c9],[ TX] [2011-0119-0000000B ] 1
          BLOCKER 5f4689ba8 5 [0x2090018][ 0x59c9],[ TX] [1018-018E-0000000B ] 0
          BLOCKED 5f4689ba8 5 [0x2090018][ 0x59c9],[ TX] [1018-018E-0000000B ] 0
          BLOCKER 5f4689a58 5 [0x2090018][ 0x59c9],[ TX] [100E-00EB-00000004 ] 0
          .............................

          This is how you interpret above information..

          Process on node 1 with xid 201C-01CA-00000004  is waiting for TX lock in mode 5 (exclusive access) which is held by process with xid 2011-0119-0000000B on node 1.

          The object involved can be found by converting hex value 0x59c9 to decimal value (i.e. 22985)  and then use dba_objects view.

          The sql involved here can be found by checking bdump or udump directory on respective node if not mentioned in trace file. Use PID to search through  tracefiles. And in case trace file is removed then set ORA 60 "event" on every node in cluster.
          i.e. event="60 trace name errorstack level 3;name systemstate level 10"

          Let me know if this information is useful or not.

          Regards,
          Dhrumil
          757-582-8955



          From: hit_gon <hit_gon@...>
          To: itsoracleteam@yahoogroups.com
          Sent: Sat, 1 May, 2010 12:35:39 AM
          Subject: [itsoracleteam] Re: Global Enqueue Services Deadlock detected

           



          --- In itsoracleteam@ yahoogroups. com, "hit_gon" <hit_gon@... > wrote:
          >
          > Hi,
          >
          > any one can help me please ?
          >
          > I'm frequently detected following error in my alert log file.
          >
          > Global Enqueue Services Deadlock detected. More info in file
          > /u001/app/oracle/ product/10. 2.0_db/db_ 1/admin/XLITEAAA /bdump/Xlitepun1 _lmd0_55822. trc.
          >
          >
          > DB Version:
          > Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bi
          > PL/SQL Release 10.2.0.3.0 - Production
          > CORE 10.2.0.3.0 Production
          > TNS for Solaris: Version 10.2.0.3.0 - Production
          > NLSRTL Version 10.2.0.3.0 - Production
          >
          > Thanks in Advance.
          >

          Hi
          Here i've given content of Xlitepun1_lmd0_ 55822.trc.

          /u001/app/oracle/ product/10. 2.0_db/db_ 1/admin/XLITEAAA /bdump/Xlitepun1 _lmd0_55822. trc
          Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production
          With the Partitioning, Real Application Clusters, OLAP and Data Mining options
          ORACLE_HOME = /u001/app/oracle/ product/10. 2.0_db/db_ 1
          System name: SunOS
          Node name: pun-svr-smaaa- 01
          Release: 5.10
          Version: Generic_118833- 33
          Machine: sun4u
          Instance name: XLITEPUN1
          Redo thread mounted by this instance: 1
          Oracle process number: 6
          Unix process pid: 55822, image: oracle@pun-svr- smaaa-01 (LMD0)

          *** 2010-04-29 12:43:59.696
          *** SERVICE NAME:() 2010-04-29 12:43:59.686
          *** SESSION ID:(5501.1) 2010-04-29 12:43:59.686
          * kjdrchkdrm: found an RM request in the request queue
          Transfer pkey 4294950913 to node 0
          Single node DRM(1) - transfer pkey 4294950913 to 0 done
          Single node DRM(1) - transfer pkey 4294950914 to 0 done
          Single node DRM(1) - transfer pkey 4294950915 to 0 done
          Single node DRM(1) - transfer pkey 4294950916 to 0 done
          Single node DRM(1) - transfer pkey 4294950917 to 0 done
          Single node DRM(1) - transfer pkey 4294950918 to 0 done
          Single node DRM(1) - transfer pkey 4294950919 to 0 done
          Single node DRM(1) - transfer pkey 4294950920 to 0 done
          Single node DRM(1) - transfer pkey 4294950921 to 0 done
          Single node DRM(1) - transfer pkey 4294950922 to 0 done
          Single node DRM(1) - transfer pkey 4294950933 to 0 done
          Single node DRM(1) - transfer pkey 4294950934 to 0 done
          Single node DRM(1) - transfer pkey 4294950935 to 0 done
          Single node DRM(1) - transfer pkey 4294950936 to 0 done
          Single node DRM(1) - transfer pkey 4294950937 to 0 done
          Single node DRM(1) - transfer pkey 4294950938 to 0 done
          Single node DRM(1) - transfer pkey 4294950939 to 0 done
          Single node DRM(1) - transfer pkey 4294950949 to 0 done
          Single node DRM(1) - transfer pkey 4294950950 to 0 done
          Single node DRM(1) - transfer pkey 4294950951 to 0 done
          Single node DRM(1) - transfer pkey 4294950952 to 0 done
          Single node DRM(1) - transfer pkey 4294950953 to 0 done
          * kjdrchkdrm: found an RM request in the request queue
          Transfer pkey 4294950976 to node 0
          Single node DRM(2) - transfer pkey 4294950976 to 0 done
          Single node DRM(2) - transfer pkey 4294950977 to 0 done
          --More--(0%)

          * kjdrchkdrm: found an RM request in the request queue
          Transfer pkey 4294951177 to node 0
          Single node DRM(5) - transfer pkey 4294951177 to 0 done
          Single node DRM(5) - transfer pkey 4294951178 to 0 done
          Single node DRM(5) - transfer pkey 4294951179 to 0 done
          Single node DRM(5) - transfer pkey 4294951180 to 0 done
          Single node DRM(5) - transfer pkey 4294951181 to 0 done
          Single node DRM(5) - transfer pkey 4294951182 to 0 done
          Single node DRM(5) - transfer pkey 4294951183 to 0 done
          Single node DRM(5) - transfer pkey 4294951184 to 0 done
          Single node DRM(5) - transfer pkey 4294951185 to 0 done
          Single node DRM(5) - transfer pkey 4294951186 to 0 done
          Single node DRM(5) - transfer pkey 4294951187 to 0 done
          Single node DRM(5) - transfer pkey 4294951188 to 0 done
          Single node DRM(5) - transfer pkey 4294951189 to 0 done
          Single node DRM(5) - transfer pkey 4294951190 to 0 done
          Single node DRM(5) - transfer pkey 4294951191 to 0 done
          Single node DRM(5) - transfer pkey 4294951192 to 0 done
          Single node DRM(5) - transfer pkey 4294951193 to 0 done
          Single node DRM(5) - transfer pkey 4294951194 to 0 done
          Single node DRM(5) - transfer pkey 4294951195 to 0 done
          Single node DRM(5) - transfer pkey 4294951196 to 0 done
          Single node DRM(5) - transfer pkey 4294951197 to 0 done
          Single node DRM(5) - transfer pkey 4294951198 to 0 done
          Single node DRM(5) - transfer pkey 4294951199 to 0 done
          Single node DRM(5) - transfer pkey 4294951200 to 0 done
          Single node DRM(5) - transfer pkey 4294951201 to 0 done
          Single node DRM(5) - transfer pkey 4294951202 to 0 done
          Single node DRM(5) - transfer pkey 4294951203 to 0 done
          Single node DRM(5) - transfer pkey 4294951204 to 0 done
          Single node DRM(5) - transfer pkey 4294951205 to 0 done
          Single node DRM(5) - transfer pkey 4294951206 to 0 done
          Single node DRM(5) - transfer pkey 4294951207 to 0 done
          Single node DRM(5) - transfer pkey 4294951208 to 0 done
          Single node DRM(5) - transfer pkey 4294951209 to 0 done
          Single node DRM(5) - transfer pkey 4294951210 to 0 done
          Single node DRM(5) - transfer pkey 4294951211 to 0 done
          Single node DRM(5) - transfer pkey 4294951212 to 0 done
          Single node DRM(5) - transfer pkey 4294951213 to 0 done
          Single node DRM(5) - transfer pkey 4294951214 to 0 done
          * kjdrchkdrm: found an RM request in the request queue

          valblk : 0x00000000000000000 000000000000000 .
          DUMP LOCAL BLOCKER: initiate state dump for DEADLOCK
          possible owner[235.27937]
          Submitting asynchronized dump request [28]
          ----------enqueue 0x5f4689ba8- --------- --------- -----
          lock version : 27079
          Owner node : 0
          grant_level : KJUSERNL
          req_level : KJUSEREX
          bast_level : KJUSERNL
          notify_func : 0
          resp : 5c67c66b0
          procp : 5f61baa70
          pid : 40613
          proc version : 4
          oprocp : 0
          opid : 0
          group lock owner : 5f9c6fec0
          possible pid : 40613
          xid : 1018-018E-0000000B
          dd_time : -523.0 secs
          dd_count : 2
          timeout : 0.0 secs
          On_timer_q? : N
          On_dd_q? : Y
          lock_state : OPENING CONVERTING
          Open Options : KJUSERDEADLOCK
          Convert options : KJUSERGETVALUE
          History : 0x1495149a
          Msg_Seq : 0x0
          res_seq : 8
          valblk : 0x00000000000000000 000000000000000 .
          DUMP LOCAL BLOCKER: initiate state dump for DEADLOCK
          possible owner[398.40613]
          Submitting asynchronized dump request [28]
          Global Wait-For-Graph( WFG) at ddTS[0.f5] :
          BLOCKED 5f46897b8 5 [0x2090018][ 0x59c9],[ TX] [201C-01CA-00000004 ] 1
          BLOCKER 5f3701920 5 [0x2090018][ 0x59c9],[ TX] [2011-0119-0000000B ] 1
          BLOCKED 5f3701920 5 [0x2090018][ 0x59c9],[ TX] [2011-0119-0000000B ] 1
          BLOCKER 5f4689ba8 5 [0x2090018][ 0x59c9],[ TX] [1018-018E-0000000B ] 0
          BLOCKED 5f4689ba8 5 [0x2090018][ 0x59c9],[ TX] [1018-018E-0000000B ] 0
          BLOCKER 5f4689a58 5 [0x2090018][ 0x59c9],[ TX] [100E-00EB-00000004 ] 0
          BLOCKED 5f5690d90 5 [0x326000d][ 0x58c3],[ TX] [100E-00EB-00000004 ] 0
          BLOCKER 5f51587d8 5 [0x326000d][ 0x58c3],[ TX] [200A-00AA-0000000B ] 1
          BLOCKED 5f468edc8 5 [0x2090018][ 0x59c9],[ TX] [200A-00AA-0000000B ] 1
          BLOCKER 5f46897b8 5 [0x2090018][ 0x59c9],[ TX] [201C-01CA-00000004 ] 1
          -bash-3.00$


        Your message has been successfully submitted and would be delivered to recipients shortly.