[OPENDJ-5582] LdapClientSocket connection leaked when handshake fails Created: 15/Oct/18  Updated: 04/Jun/20  Resolved: 29/Oct/18

Status: Done
Project: OpenDJ
Component/s: core apis
Affects Version/s: 6.0.0, 5.5.0
Fix Version/s: 6.5.0

Type: Bug Priority: Major
Reporter: C-Weng C Assignee: Yannick Lecaillez
Resolution: Fixed Votes: 0
Labels: Verified, release-notes

Attachments: Java Source File Testcase.java    
Issue Links:
Backport
is backported by OPENDJ-5674 Backport OPENDJ-5582: LdapClientSocke... Done
is backported by OPENDJ-6296 Backport OPENDJ-5582: LdapClientSocke... Done
is backported by OPENDJ-5655 Backport OPENDJ-5582: LdapClientSocke... QA Backlog
Duplicate
is duplicated by OPENAM-14326 Incorrect Password with LDAP Decision... Closed
Regression
caused OPENAM-13740 File descriptor / Connection leak wh... Resolved
Relates
relates to OPENAM-13740 File descriptor / Connection leak wh... Resolved
is related to OPENDJ-5621 QA: test connection leak Done
Epic Link: Bugs 6.5
Story Points: 1
Support Ticket IDs:

 Description   

Problem

When the DJ LoadBalancer implementation is used with the following with a pre-binded authentication user, there seems to be failure cases that lead to continuous connection leak.
As a client OPENDJ sdk usage there seems to be issues trying to use the newFailoverLoadBalancer+newFailoverLoadBalancer+LdapClient.

       Options options = Options.defaultOptions();
       if (username != null) {
           options = options.set(AUTHN_BIND_REQUEST,
                     newSimpleBindRequest(username, password));
       }
       List<LdapClient> ldapClients = new ArrayList<>();
       LdapClient primaryClient = newFixedConnectionPool(
                 newLdapClient(serverName, port, options), 1);
       ldapClients.add(primaryClient);
       LdapClient LBClient = newFailoverLoadBalancer(ldapClients,options);
       LdapConnectionFactory factory = new LdapConnectionFactory(LBClient);
       Connection c = factory.getConnection();
      // do a search and close all (obviously this connection is not usable
      // but thee is background leakage that grows non-stop

When using the above with a wrong username/or password there is a background retry of these connections that causes leak

io.reactivex.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)
	at io.reactivex.Single.subscribe(Single.java:3096)
	at io.reactivex.Single.subscribe(Single.java:3082)
	at org.forgerock.opendj.ldap.LoadBalancer$MonitoredLdapClient.checkIfAvailable(LoadBalancer.java:296)
	at org.forgerock.opendj.ldap.LoadBalancer$MonitoredLdapClient.access$1700(LoadBalancer.java:259)
	at org.forgerock.opendj.ldap.LoadBalancer.checkIfServersAreOnline(LoadBalancer.java:417)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)

Run the attached testcase, but monitor the netstat or lsof to see that connection growth happens to DJ.

Affected versions

tested on DJ6. Tested on DJ6.5 (client libs) but no improvement

Testcase

Use the attached testcase and monitoring the connection to DJ even after the connection is closed.

Expected
No connection leak

Currently
Connection growth leak even if there is no activity. The background checkIfAvailable in LoadBalancer is repeated get call and leaks connection regularly

Analysis

If there is no wrong password or if no newFailoverLoadBalancer is used this issue is not seen.

Side concerns: It has been reported that this is one of the cases of leakage but there seems to be reported that even there is no issue with password, it is felt that if there is some exceptions, it is possible that connection leakage happens (so hopefully the solution here will also fix other related causes).



 Comments   
Comment by Matthew Swift [ 15/Oct/18 ]

Thanks for the detailed report. Resource leaks are never a good idea, especially file-descriptors. I've triaged this bug for 6.5.

Comment by Yannick Lecaillez [ 16/Oct/18 ]

The leak happens each time an error occurs during connection handshake phase either in StartTLS (LDAP_CLIENT_SSL_USE_STARTTLS), Bind (LDAP_CLIENT_AUTHN_BIND_REQUEST*) or initial heartbeat (LDAP_CLIENT_HEARTBEAT_ENABLED).
It could also be a candidate for backport.

Comment by Chris Ridd [ 20/Mar/19 ]

I've removed 4.0.0 (DJ 5.0.0) from the affected versions because the leaked class does not exist in that version, so this exact bug cannot affect 4.0.0.

Comment by Petr Matej [X] (Inactive) [ 09/Jul/19 ]

Verified on 6.5.0

Generated at Wed Sep 30 04:02:26 UTC 2020 using Jira 7.13.12#713012-sha1:6e07c38070d5191bbf7353952ed38f111754533a.