Uploaded image for project: 'OpenDJ'
  1. OpenDJ
  2. OPENDJ-7776

Active dynamic group search threads cause stop-ds command to time out and fail to stop a server

    XMLWordPrintable

Details

    • Bug
    • Status: Done
    • Major
    • Resolution: Fixed
    • 6.5.2
    • 7.1.0
    • None

    Description

      When stopping DS using stop-ds, it becomes unresponsive and times out

      $ ./stop-ds
      Stopping Server...
      The timeout of '200' seconds to start the server has been reached.
      You can use the argument '--timeout' to increase this timeout
      An error occurred while parsing the command-line arguments:
      A port number must be specified to connect to the server
      

      Jstacks show a number of threads of type:

      "Dynamic Group Search Thread cn=XXXX,ou=YYYY,dc=com" #12365 prio=5 os_prio=0 tid=0x00007f5160076000 nid=0x112a waiting on condition [0x00007f51445cd000]
      java.lang.Thread.State: TIMED_WAITING (parking)
          at sun.misc.Unsafe.park(Native Method)
          - parking to wait for <0x00000000b50b6280> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
          at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
          at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
          at java.util.concurrent.LinkedBlockingQueue.offer(LinkedBlockingQueue.java:385)
          at org.opends.server.extensions.DynamicGroupMemberList.addResult(DynamicGroupMemberList.java:232)
          at org.opends.server.extensions.DynamicGroupSearchThread.handleEntry(DynamicGroupSearchThread.java:135)
          at org.opends.server.protocols.internal.AbstractInternalClientConnection$InternalConnection$$Lambda$114/8996952.accept(Unknown Source)
          at io.reactivex.internal.subscribers.LambdaSubscriber.onNext(LambdaSubscriber.java:65)
          at io.reactivex.internal.operators.flowable.FlowableMap$MapSubscriber.onNext(FlowableMap.java:69)
          at io.reactivex.internal.operators.flowable.FlowableFilter$FilterSubscriber.tryOnNext(FlowableFilter.java:74)
          at io.reactivex.internal.operators.flowable.FlowableFilter$FilterSubscriber.onNext(FlowableFilter.java:52)
          at io.reactivex.internal.operators.flowable.FlowablePublish$PublishSubscriber.dispatch(FlowablePublish.java:528)
          at io.reactivex.internal.operators.flowable.FlowablePublish$PublishSubscriber.onNext(FlowablePublish.java:229)
          at io.reactivex.internal.operators.flowable.FlowableOnErrorNext$OnErrorNextSubscriber.onNext(FlowableOnErrorNext.java:69)
          at io.reactivex.internal.operators.flowable.FlowableDoOnEach$DoOnEachSubscriber.onNext(FlowableDoOnEach.java:92)
          at io.reactivex.internal.operators.flowable.FlowableMap$MapSubscriber.onNext(FlowableMap.java:69)
          at io.reactivex.internal.operators.flowable.FlowableDoFinally$DoFinallySubscriber.onNext(FlowableDoFinally.java:85)
          at io.reactivex.internal.operators.flowable.FlowableDoOnEach$DoOnEachSubscriber.onNext(FlowableDoOnEach.java:92)
          at org.forgerock.opendj.server.core.RequestState.onNext(RequestState.java:128)
          at org.forgerock.opendj.server.core.RequestState.onNext(RequestState.java:45)
          at io.reactivex.internal.operators.flowable.FlowableCreate$NoOverflowBaseAsyncEmitter.onNext(FlowableCreate.java:395)
          at org.opends.server.protocols.internal.AbstractInternalClientConnection.sendSearchEntry(AbstractInternalClientConnection.java:544)
          at org.opends.server.protocols.internal.InternalClientConnection.sendSearchEntry(InternalClientConnection.java:42)
          at org.opends.server.core.SearchOperation.sendSearchEntry(SearchOperation.java:715)
          at org.opends.server.core.SearchOperation.returnEntry(SearchOperation.java:479)
          at org.opends.server.backends.pluggable.EntryContainer.returnEntryOrStop(EntryContainer.java:1166)
          at org.opends.server.backends.pluggable.EntryContainer.searchIndexed(EntryContainer.java:1370)
          at org.opends.server.backends.pluggable.EntryContainer.access$1700(EntryContainer.java:152)
          at org.opends.server.backends.pluggable.EntryContainer$1.run(EntryContainer.java:765)
          at org.opends.server.backends.pluggable.EntryContainer$1.run(EntryContainer.java:629)
          at org.opends.server.backends.jeb.JEStorage.read(JEStorage.java:926)
          at org.opends.server.backends.pluggable.TracedStorage.read(TracedStorage.java:414)
          at org.opends.server.backends.pluggable.EntryContainer.search(EntryContainer.java:629)
          at org.opends.server.backends.pluggable.BackendImpl.search(BackendImpl.java:466)
          at org.opends.server.core.SearchOperation.processSearch(SearchOperation.java:888)
          at org.opends.server.core.SearchOperation.processLocalSearch(SearchOperation.java:836)
          at org.opends.server.core.SearchOperation.run(SearchOperation.java:764)
          at org.opends.server.protocols.internal.AbstractInternalClientConnection.handleRequest(AbstractInternalClientConnection.java:365)
          at org.opends.server.protocols.internal.AbstractInternalClientConnection.lambda$handle$0(AbstractInternalClientConnection.java:348)
          at org.opends.server.protocols.internal.AbstractInternalClientConnection$$Lambda$109/415906914.subscribe(Unknown Source)
          at io.reactivex.internal.operators.flowable.FlowableCreate.subscribeActual(FlowableCreate.java:72)
          at io.reactivex.Flowable.subscribe(Flowable.java:13234)
          at io.reactivex.Flowable.subscribe(Flowable.java:13180)
          at io.reactivex.internal.operators.flowable.FlowableLift.subscribeActual(FlowableLift.java:49)
          at io.reactivex.Flowable.subscribe(Flowable.java:13234)
          at io.reactivex.internal.operators.flowable.FlowableDoOnEach.subscribeActual(FlowableDoOnEach.java:50)
          at io.reactivex.Flowable.subscribe(Flowable.java:13234)
          at io.reactivex.internal.operators.flowable.FlowableDoFinally.subscribeActual(FlowableDoFinally.java:47)
          at io.reactivex.Flowable.subscribe(Flowable.java:13234)
          at io.reactivex.internal.operators.flowable.FlowableMap.subscribeActual(FlowableMap.java:38)
          at io.reactivex.Flowable.subscribe(Flowable.java:13234)
          at io.reactivex.internal.operators.flowable.FlowableDoOnEach.subscribeActual(FlowableDoOnEach.java:50)
          at io.reactivex.Flowable.subscribe(Flowable.java:13234)
          at io.reactivex.internal.operators.flowable.FlowableOnErrorNext.subscribeActual(FlowableOnErrorNext.java:39)
          at io.reactivex.Flowable.subscribe(Flowable.java:13234)
          at io.reactivex.internal.operators.flowable.FlowablePublish.connect(FlowablePublish.java:130)
          at io.reactivex.flowables.ConnectableFlowable.connect(ConnectableFlowable.java:64)
          at org.opends.server.protocols.internal.AbstractInternalClientConnection$InternalConnection.search(AbstractInternalClientConnection.java:146)
          at org.opends.server.extensions.DynamicGroupSearchThread.run(DynamicGroupSearchThread.java:105)
      

      Full jstack file attached.

      Attachments

        Issue Links

          Activity

            People

              ondrej.fuchsik Ondrej Fuchsik
              akhil.kommadath Akhil Kommadath
              Jean-Noël Rouvignac Jean-Noël Rouvignac
              Ondrej Fuchsik Ondrej Fuchsik
              Votes:
              3 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: