Uploaded image for project: 'OpenAM'
  1. OpenAM
  2. OPENAM-14078

RetryTask can block notification processing for an extended period of time

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.5.0
    • Fix Version/s: 6.5.0.1, 6.5.1, 7.0.0
    • Component/s: idrepo
    • Labels:
      None
    • Needs backport:
      Yes
    • Needs QA verification:
      No
    • Functional tests:
      No
    • Are the reproduction steps defined?:
      Yes and I used the same an in the description

      Description

      Bug description

      When running the functional tests, at times, the tests have been failing with failed to create user errors, and read timeouts. Upon investigation I've seen threads like these:

      "http-bio-127.0.0.1-8080-exec-15" #524 daemon prio=5 os_prio=0 tid=0x00007fb8c00596a0 nid=0x2fcaa waiting for monitor entry [0x00007fb91e9b8000]
         java.lang.Thread.State: BLOCKED (on object monitor)
              at org.forgerock.openam.idrepo.ldap.DJLDAPv3Repo.addListener(DJLDAPv3Repo.java:2315)
              - waiting to lock <0x00000000b8a5b968> (a java.util.HashMap)
              at org.forgerock.openam.idm.IdRepoAuditor.addListener(IdRepoAuditor.java:359)
              at com.sun.identity.idm.server.IdRepoPluginsCache.constructIdRepoPlugin(IdRepoPluginsCache.java:476)
              at com.sun.identity.idm.server.IdRepoPluginsCache.getIdRepoPlugins(IdRepoPluginsCache.java:156)
              - locked <0x00000000b6b74a58> (a java.util.HashMap)
              at com.sun.identity.idm.server.IdServicesImpl.getSupportedOperations(IdServicesImpl.java:2304)
              at com.sun.identity.idm.AMIdentityRepository.getAllowedIdOperations(AMIdentityRepository.java:184)
              at com.sun.identity.idsvcs.opensso.IdentityServicesImpl.isOperationSupported(IdentityServicesImpl.java:961)
              at com.sun.identity.idsvcs.opensso.IdentityServicesImpl.create(IdentityServicesImpl.java:148)
      "smIdmThreadPool-9" #337 daemon prio=5 os_prio=0 tid=0x00007fb8e0a3be70 nid=0x8ed waiting for monitor entry [0x00007fb7756db000]
         java.lang.Thread.State: BLOCKED (on object monitor)
              at org.forgerock.openam.idrepo.ldap.DJLDAPv3Repo.removeListener(DJLDAPv3Repo.java:2356)
              - waiting to lock <0x00000000b8a5b968> (a java.util.HashMap)
              at org.forgerock.openam.idm.IdRepoAuditor.removeListener(IdRepoAuditor.java:364)
              at com.sun.identity.idm.server.IdRepoPluginsCache$ShutdownIdRepoPlugin.run(IdRepoPluginsCache.java:673)
              at org.forgerock.openam.audit.context.AuditRequestContextPropagatingRunnable.run(AuditRequestContextPropagatingRunnable.java:34)
      

      The threads were ultimately blocked by:

      "SystemTimerPool" #535 prio=5 os_prio=0 tid=0x00007fb6f0015940 nid=0x57fe waiting on condition [0x00007fb55aaf3000]
         java.lang.Thread.State: TIMED_WAITING (sleeping)
              at java.lang.Thread.sleep(Native Method)
              at com.iplanet.services.ldap.event.LDAPv3PersistentSearch$RetryTask.run(LDAPv3PersistentSearch.java:441)
              at com.sun.identity.common.TimerPool$WorkerThread.run(TimerPool.java:469)
      

      How to reproduce the issue

      No clear reproduction steps, but the stacktraces suggest that a psearch has to fail (due to connection errors) and get into the long waiting phase, then try to remove the data store (or realm), and try to perform an operation in a different (or new) realm.

      Expected behaviour

      There is no slowdown in the system.

      Current behaviour

      The thread is blocked for a long time, it is not clear whether it even terminates at one point.

        Attachments

          Activity

            People

            • Assignee:
              peter.major Peter Major [X] (Inactive)
              Reporter:
              peter.major Peter Major [X] (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: