Uploaded image for project: 'OpenDJ'
  1. OpenDJ
  2. OPENDJ-1130

Connect Error when stopping all the servers from a replication topology at the same time

    XMLWordPrintable

Details

    Description

      Found with functional tests using revision 9467.

      In our tests, we configure a replication topology with 2 DS/RS servers.
      At the end of the test, we stop all the servers in parallel and we get the following errors:

      $ ./opendj-1/bin/stop-ds -X -h localhost -p 11444 -D cn=myself -w password
      ERROR: Cannot establish a connection to the Directory Server
      localhost:11444. Verify that the server is running and that the provided
      credentials are valid. Details: Connect Error
      $ ./opendj-2/bin/stop-ds -X -h localhost -p 22444 -D cn=myself -w password
      ERROR: Cannot establish a connection to the Directory Server
      localhost:22444. Verify that the server is running and that the provided
      credentials are valid. Details: Connect Error
      

      => the servers are not stopped

      Here the traces from the errors log:

      [29/Aug/2013:15:24:10 +0200] category=BACKEND severity=NOTICE msgID=9896349 msg=Shutdown task 01ebee87-4a8a-45d0-894c-51198444adb3 started execution
      [29/Aug/2013:15:24:10 +0200] category=BACKEND severity=NOTICE msgID=9896350 msg=Shutdown task 01ebee87-4a8a-45d0-894c-51198444adb3 finished execution
      [29/Aug/2013:15:24:10 +0200] category=CORE severity=NOTICE msgID=458891 msg=The Directory Server has sent an alert notification generated by class org.open
      ds.server.core.DirectoryServer (alert type org.opends.server.DirectoryServerShutdown, alert ID 458893):  The Directory Server has started the shutdown proc
      ess.  The shutdown was initiated by an instance of class org.opends.server.tasks.ShutdownTaskThread and the reason provided for the shutdown was The Direct
      ory Server shutdown process has been initiated by task ds-task-id=01ebee87-4a8a-45d0-894c-51198444adb3,cn=Scheduled Tasks,cn=Tasks
      [29/Aug/2013:15:24:11 +0200] category=CORE severity=FATAL_ERROR msgID=327820 msg=An uncaught exception during processing for thread Shutdown Task Thread ha
      s caused it to terminate abnormally.  The stack trace for that exception is:  java.lang.IllegalStateException: Unclosed Database: 13782 cn=admin
      Unclosed Database: 31588 cn=admin
      Databases left open: 2
      
        com.sleepycat.je.Environment.close(Environment.java:384)
        org.opends.server.replication.server.changelog.je.ReplicationDbEnv.shutdown(ReplicationDbEnv.java:381)
        org.opends.server.replication.server.ReplicationServer.shutdown(ReplicationServer.java:843)
        org.opends.server.replication.plugin.ReplicationServerListener.shutdown(ReplicationServerListener.java:114)
        org.opends.server.replication.plugin.MultimasterReplication.finalizeSynchronizationProvider(MultimasterReplication.java:598)
        org.opends.server.core.DirectoryServer.shutDown(DirectoryServer.java:8052)
        org.opends.server.tasks.ShutdownTaskThread.run(ShutdownTaskThread.java:82)
      
      [29/Aug/2013:15:24:11 +0200] category=CORE severity=NOTICE msgID=458891 msg=The Directory Server has sent an alert notification generated by class org.open
      ds.server.api.DirectoryThread (alert type org.opends.server.UncaughtException, alert ID 327820):  An uncaught exception during processing for thread Shutdo
      wn Task Thread has caused it to terminate abnormally.  The stack trace for that exception is:  java.lang.IllegalStateException: Unclosed Database: 13782 cn
      =admin
      Unclosed Database: 31588 cn=admin
      Databases left open: 2
      
        com.sleepycat.je.Environment.close(Environment.java:384)
        org.opends.server.replication.server.changelog.je.ReplicationDbEnv.shutdown(ReplicationDbEnv.java:381)
        org.opends.server.replication.server.ReplicationServer.shutdown(ReplicationServer.java:843)
        org.opends.server.replication.plugin.ReplicationServerListener.shutdown(ReplicationServerListener.java:114)
        org.opends.server.replication.plugin.MultimasterReplication.finalizeSynchronizationProvider(MultimasterReplication.java:598)
        org.opends.server.core.DirectoryServer.shutDown(DirectoryServer.java:8052)
        org.opends.server.tasks.ShutdownTaskThread.run(ShutdownTaskThread.java:82)
      
      [29/Aug/2013:15:24:11 +0200] category=SYNC severity=SEVERE_ERROR msgID=-1 msg=Error during the Replication Server database trimming or flush process. The C
      hangelog service is going to shutdownCan't open a cursor Database was closed. (Database.java:1863 Database.java:677 ReplicationDB.java:469 DbHandler.java:4
      31 DbHandler.java:358 Thread.java:722)
      [29/Aug/2013:15:24:11 +0200] category=SYNC severity=SEVERE_ERROR msgID=-1 msg=Error during the Replication Server database trimming or flush process. The C
      hangelog service is going to shutdownCan't open a cursor Database was closed. (Database.java:1863 Database.java:677 ReplicationDB.java:469 DbHandler.java:4
      31 DbHandler.java:358 Thread.java:722)
      [29/Aug/2013:15:24:40 +0200] category=CORE severity=SEVERE_WARNING msgID=131185 msg=Worker Thread 0 was interrupted while waiting for new work:  java.lang.
      InterruptedException.  This should not happen, but the thread will resume waiting for new work so there should be no adverse effects
      [29/Aug/2013:15:24:40 +0200] category=CORE severity=SEVERE_WARNING msgID=131185 msg=Worker Thread 2 was interrupted while waiting for new work:  java.lang.
      InterruptedException.  This should not happen, but the thread will resume waiting for new work so there should be no adverse effects
      [29/Aug/2013:15:24:40 +0200] category=CORE severity=SEVERE_WARNING msgID=131185 msg=Worker Thread 6 was interrupted while waiting for new work:  java.lang.
      InterruptedException.  This should not happen, but the thread will resume waiting for new work so there should be no adverse effects
      [29/Aug/2013:15:24:40 +0200] category=CORE severity=SEVERE_WARNING msgID=131185 msg=Worker Thread 8 was interrupted while waiting for new work:  java.lang.
      InterruptedException.  This should not happen, but the thread will resume waiting for new work so there should be no adverse effects
      ...
      

      Attachments

        Activity

          People

            cjr Chris Ridd
            csovant Christophe Sovant
            Christophe Sovant Christophe Sovant
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: