Uploaded image for project: 'OpenAM'
  1. OpenAM
  2. OPENAM-9915

Shutdown can hang for ~2 minutes

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 14.0.0
    • Fix Version/s: 14.0.0
    • Component/s: CTS, session
    • Labels:
    • Target Version/s:
    • Rank:
      1|hzpb7j:
    • Sprint:
      AM Sustaining Sprint 32
    • Story Points:
      8

      Description

      When trying to shut down OpenAM, the following thread can be seen in jstack:

      "ServerService Thread Pool -- 98" prio=5 tid=0x00007fd628c8c000 nid=0x26007 waiting on condition [0x0000700006309000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007f79267c0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.forgerock.openam.cts.impl.queue.AsyncResultHandler.getResults(AsyncResultHandler.java:86)
	at org.forgerock.openam.cts.monitoring.impl.queue.TokenMonitoringResultHandler.getResults(TokenMonitoringResultHandler.java:50)
	at org.forgerock.openam.cts.monitoring.impl.queue.TokenMonitoringResultHandler.getResults(TokenMonitoringResultHandler.java:27)
	at org.forgerock.openam.cts.CTSPersistentStoreImpl.update(CTSPersistentStoreImpl.java:105)
	at org.forgerock.openam.session.service.SessionPersistentStore.save(SessionPersistentStore.java:114)
	at org.forgerock.openam.session.service.SessionAccessManager.save(SessionAccessManager.java:361)
	at org.forgerock.openam.session.service.SessionAccessManager.update(SessionAccessManager.java:350)
	at org.forgerock.openam.session.service.SessionAccessManager.notifyUpdate(SessionAccessManager.java:383)
	at com.iplanet.dpro.session.service.InternalSession.notifyPersistenceManager(InternalSession.java:1244)
	at com.iplanet.dpro.session.service.InternalSession.setLatestAccessTime(InternalSession.java:950)
	at com.iplanet.dpro.session.operations.strategies.LocalOperations.getSessionInfo(LocalOperations.java:204)
	at com.iplanet.dpro.session.operations.strategies.LocalOperations.refresh(LocalOperations.java:138)
	at com.iplanet.dpro.session.monitoring.MonitoredOperations.refresh(MonitoredOperations.java:67)
	at com.iplanet.dpro.session.Session.doRefresh(Session.java:764)
	at com.iplanet.dpro.session.Session.access$000(Session.java:84)
	at com.iplanet.dpro.session.Session$1.run(Session.java:740)
	at com.sun.identity.session.util.RestrictedTokenContext.doUsing(RestrictedTokenContext.java:81)
	at com.iplanet.dpro.session.Session.refresh(Session.java:737)
	at org.forgerock.openam.session.SessionCache.getSession(SessionCache.java:324)
	at org.forgerock.openam.session.SessionCache.getSession(SessionCache.java:262)
	at org.forgerock.openam.session.SessionCache.getSession(SessionCache.java:247)
	at com.iplanet.sso.providers.dpro.SSOProviderImpl.destroyToken(SSOProviderImpl.java:343)
	at com.iplanet.sso.SSOTokenManager.destroyToken(SSOTokenManager.java:490)
	at com.sun.identity.security.AdminTokenAction.resetInstance(AdminTokenAction.java:182)
	at com.sun.identity.security.AdminTokenAction.reset(AdminTokenAction.java:176)
	at com.sun.identity.security.AdminTokenAction$1.shutdown(AdminTokenAction.java:131)
	at com.sun.identity.common.ShutdownManager.shutdown(ShutdownManager.java:218)
	at com.sun.identity.common.ShutdownServletContextListener.contextDestroyed(ShutdownServletContextListener.java:51)
	at io.undertow.servlet.core.ApplicationListeners.contextDestroyed(ApplicationListeners.java:185)
	at io.undertow.servlet.core.DeploymentImpl.destroy(DeploymentImpl.java:224)
	at io.undertow.servlet.core.DeploymentManagerImpl.undeploy(DeploymentManagerImpl.java:578)
	at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.stopContext(UndertowDeploymentService.java:145)
	at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$2.run(UndertowDeploymentService.java:121)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at org.jboss.threads.JBossThread.run(JBossThread.java:320)

      and the shutdown takes quite a long while.

      I believe this is happening because AdminTokenAction#reset tries to change session state during container shutdown and most likely the CTS connections are already closed, hence the implementation just waits for 2 minutes waiting for a result that is never going to arrive.

        Attachments

          Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. OPENAM-5980 CTS async queue can cause threads to wait for a long time

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

            Activity

              People

              Assignee:
              markdr Mark de Reeper
              Reporter:
              peter.major Peter Major [X] (Inactive)
              QA Assignee:
              Filip Kubáň [X] Filip Kubáň [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: