Uploaded image for project: 'OpenDJ'
  1. OpenDJ
  2. OPENDJ-7169

Apparent issue with failover functionality in rest2ldap

    Details

    • Type: Bug
    • Status: Done
    • Priority: Blocker
    • Resolution: Not a defect
    • Affects Version/s: 7.0.0
    • Fix Version/s: 7.0.0
    • Component/s: core apis
    • Labels:
      None

      Description

      Jake Feasel and I did testing on a GKE-deployed IDM instance with two DS instances, each configured as both replication and directory servers, in active-passive setup (one specified in primaryLdapServers, one in secondaryLdapServers). Load was generated with JMeter at ~60 user creates per second. Then the pod hosting the primary DS instance was killed. JMeter recorded ~10 failures following the initial pod kill, and again another ~10 failures when Kubernetes restored the pod. These failures were reproducible.

      JMeter reported a NoHttpResponseException (stack trace below). Note that no logs were recorded corresponding to the failures in IDM. This means that either:

      1. something other than a ResourceException was returned in the Promise returned from the rest2ldap invocations, or an unchecked exception was thrown,
      2. or the calls to rest2ldap simply did not return. 

      If it is important to make this distinction, I could surround the invocation of the repo layer for managed user creations with a try-finally, and increment an AtomicInteger prior to the repo layer invocation, and decrement it in the finally, and then reproduce the issue with Jake. A zero-valued AtomicInteger would indicate an unexpected exception, and a greater-than-0-valued AtomicInteger would indicate a repo-layer/rest2ldap call which simply did not return.

       

      org.apache.http.NoHttpResponseException: jake.iam.forgeops.com:443 failed to respond
       at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
       at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
       at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
       at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:286)
       at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:257)
       at org.apache.jmeter.protocol.http.sampler.hc.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:199)
       at org.apache.jmeter.protocol.http.sampler.MeasuringConnectionManager$MeasuredConnection.receiveResponseHeader(MeasuringConnectionManager.java:212)
       at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
       at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
       at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:684)
       at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
       at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
       at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
       at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.executeRequest(HTTPHC4Impl.java:695)
       at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:454)
       at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:74)
       at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1189)
       at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1178)
       at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:498)
       at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:424)
       at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:255)
       at java.lang.Thread.run(Thread.java:748)
      

        Attachments

          Activity

            People

            • Assignee:
              ylecaillez Yannick Lecaillez
              Reporter:
              dhogan Dirk Hogan
              Dev Assignee:
              Yannick Lecaillez
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: