TLSv1.3 support was introduced in JDK11. It dramatically changes the TLS handshake protocol, especially for cases where client authentication is performed:
- client handshake -> server
- server handshake -> client (does not include list of acceptable CAs)
- install negotiate security layer
- handshake completed
- client sends client certificate along with initial application data packet (e.g. bind) to server
From an client application point of view the connection attempt succeeds at step 4 despite having not sent the client certificate. The client then sends a bind request and implicitly its TLS client cert. The server may reject the cert without processing the application data packet. This triggers an SSLException in the server which causes the server to close the connection. IIRC, the processing is quite racey and may result in either a server close or protocol error client-side, neither of which help users diagnose the problem.
This issue may be closed once:
- the client/server handles client auth failures in a deterministic fashion, i.e. always generate the same error client side
- the failure is exposed to the end-user with a more helpful error message or code that may help them diagnose the problem more easily.
See the commented out test in org.opends.server.protocols.ldap.TestLDAPConnectionHandler#trustStoreManagers