Uploaded image for project: 'OpenAM Agents'
  1. OpenAM Agents
  2. AMAGENTS-662

Slow or hanging startup observed in shared memory locking

    XMLWordPrintable

    Details

      Description

      This problem manifests in IIS sites being reported with very low memory compared to other agent enabled sites.

      On investigation then one or more sites are waitingForSingleObject where there is no other locker in other dump files. This means that one of the processes had the wait_abandoned return and then continued, but something has gone wrong with the locking mechanism.

      3+ sites are needed for this

      Start one site and lock the config

      Start two other sites and while the lock is in the config then kill site 1.

      ===

      Here you can see from debug diag with spaced dump files - note that though the thread is taking up no time, it is not allowing this to proceed. In the dump file there was no limit to the time this could take (hours)

      Thread 20124

      General Operation Info

      Type System.Collections.Generic.Dictionary`2[System.Int32,System.String]
      Dumps present All dumps
      Elapsed Time
      Minimum:
      Maximum:
      At least 00:00:13
      (unknown)

       

      Thread Info

      Entry point ntdll!TppWorkerThread
      System ID 20124 (0x4E9C)
      Create time 6/14/2017 12:21:41 AM

       

       

      Call Stacks

       

       

      Dump 1

      ntdll!ZwWaitForSingleObject
      KERNELBASE!WaitForSingleObjectEx
      mod_iis_openam_64!am_shm_create
      mod_iis_openam_64!am_configuration_init
      mod_iis_openam_64!RegisterModule
      iiscore!VIRTUAL_MODULE::RegisterModule
      iiscore!W3_SERVER::LoadModulesFromConfig
      iiscore!W3_SERVER::InitializeGlobalModules
      iiscore!W3_SERVER::Initialize
      iiscore!IISCORE_PROTOCOL_MANAGER::InitializeGlobals
      iiscore!IISCORE_PROTOCOL_MANAGER::PreloadApplication
      w3wphost!WP_IPM::HandlePreloadApplications
      w3wphost!WP_IPM::AcceptMessage
      iisutil!IPM_MESSAGE_PIPE::MessagePipeCompletion
      ntdll!RtlpTpWaitCallback
      ntdll!TppWaitpExecuteCallback
      ntdll!TppWorkerThread
      kernel32!BaseThreadInitThunk
      ntdll!RtlUserThreadStart

       

      User Time 00:00:00.015
      Kernel time 00:00:00.015

      Dump 2

      ntdll!ZwWaitForSingleObject
      KERNELBASE!WaitForSingleObjectEx
      mod_iis_openam_64!am_shm_create
      mod_iis_openam_64!am_configuration_init
      mod_iis_openam_64!RegisterModule
      iiscore!VIRTUAL_MODULE::RegisterModule
      iiscore!W3_SERVER::LoadModulesFromConfig
      iiscore!W3_SERVER::InitializeGlobalModules
      iiscore!W3_SERVER::Initialize
      iiscore!IISCORE_PROTOCOL_MANAGER::InitializeGlobals
      iiscore!IISCORE_PROTOCOL_MANAGER::PreloadApplication
      w3wphost!WP_IPM::HandlePreloadApplications
      w3wphost!WP_IPM::AcceptMessage
      iisutil!IPM_MESSAGE_PIPE::MessagePipeCompletion
      ntdll!RtlpTpWaitCallback
      ntdll!TppWaitpExecuteCallback
      ntdll!TppWorkerThread
      kernel32!BaseThreadInitThunk
      ntdll!RtlUserThreadStart

       

      User Time 00:00:00.015 ()
      Kernel time 00:00:00.015 ()

      Dump 3

      ntdll!ZwWaitForSingleObject
      KERNELBASE!WaitForSingleObjectEx
      mod_iis_openam_64!am_shm_create
      mod_iis_openam_64!am_configuration_init
      mod_iis_openam_64!RegisterModule
      iiscore!VIRTUAL_MODULE::RegisterModule
      iiscore!W3_SERVER::LoadModulesFromConfig
      iiscore!W3_SERVER::InitializeGlobalModules
      iiscore!W3_SERVER::Initialize
      iiscore!IISCORE_PROTOCOL_MANAGER::InitializeGlobals
      iiscore!IISCORE_PROTOCOL_MANAGER::PreloadApplication
      w3wphost!WP_IPM::HandlePreloadApplications
      w3wphost!WP_IPM::AcceptMessage
      iisutil!IPM_MESSAGE_PIPE::MessagePipeCompletion
      ntdll!RtlpTpWaitCallback
      ntdll!TppWaitpExecuteCallback
      ntdll!TppWorkerThread
      kernel32!BaseThreadInitThunk
      ntdll!RtlUserThreadStart

       

      User Time 00:00:00.015 ()
      Kernel time 00:00:00.015 ()

        Attachments

          Activity

            People

            mareks Mareks Malnacs
            alex.levin@forgerock.com Alex Levin
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: