We are using LDAPConnectionPool feature of the SDK to maintain the connections. When an operation needs to be performed we are getting a connection from the pool and performing the action. We observed that we are getting the LDAPException 85 occasionally. Based on the documentation provided, the suggestion is to invoke a random search with the connection object before making the actual operation which is having performance impact. To avoid this, we are tracking the exception and performing 1 retry if timeout happens. Before performing the retry we are invoking pool.releaseDefunctConnection(conn) and getting a fresh connection from the pool. We observed this happened in most of the cases but as we don't know which connection will be pulled from the pool, we are are ending up with LDAPException 85 in some of the retry attempts. Is there a way we can release all the invalid connections before an operation needs to be executed in the connection pool without impacting the performance?
We are using unboundID 6.0.1 version
Last edit: Jyotir 2021-10-13
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If you're getting a result code of 85, then that means that the server didn't return a response within the maximum allowed response time. You can set default response timeouts in general and on a per-operation-type basis using the LDAPConnectionOptions class, and you can also set a response timeout on a per-operation basis by using the setResponseTimeoutMillis method on the request. If the problem is that the timeout is legitimately too short for the operation you're processing, then you can use one of those methods for adjusting it.
However, if the problem is that the connection pool has a set of connections that it thinks are valid but actually aren't and can't be used, then there are a couple of main causes for that:
The machine running the server to which the connection was established crashed, froze, or lost power in a way that prevented it from sending a TCP notification that the connection is no longer valid.
Something in the network path between the client and the server has silently dropped the connection without sending an appropriate TCP notification to either end of the connection. This is unfortunately a common occurrence.
If something like that has happened, then it's unfortunately not possible to passively detect that the connection is no longer valid in a timely manner. The best option that is available to us is TCP keep-alive, and while we do make use of that, it can be hours before the connection is actually considered invalid. As such, the best way of identifying that in a timely manner is to use a combination of health checking and maximum connection age.
The maximum connection age feature allows the LDAP SDK to automatically close and re-establish connections after they've been connected for longer than a specified length of time. That can help prevent them from being seen as "stale" by networking equipment, and it addresses the second class of problems I outlined above.
For the first class of problems, the best way to address that is with health checking. Connection pools can be configured with health checks that can perform a specified set of processing on certain events, including:
Checking to ensure that a newly established connection is valid
Checking to ensure that a connection is valid when checking it out of the pool
Checking to ensure that a connection is valid when releasing it back to the pool
Periodic checks in the background to ensure that all connections not actively in use are still valid
The GetEntryLDAPConnectionPoolHealthCheck class provides a simple implementation that tries to retrieve a specified entry, and it will consider the connection invalid if it gets an error while trying to communicate with the server, if the entry doesn't appear to exist or can't be retrieved, or if it takes too long to get a response from the server. If you want to use this or a similar health check, then I'd probably start with just using background health checks for connections not in use, since that doesn't have a significant impact on performance. Invoking a health check on every checkout or release is more expensive because the thread doing the checking out or releasing has to wait on the health check, and the attempt to use the connection for whatever operation you want to process will ultimately do the same basic thing.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
We are using LDAPConnectionPool feature of the SDK to maintain the connections. When an operation needs to be performed we are getting a connection from the pool and performing the action. We observed that we are getting the LDAPException 85 occasionally. Based on the documentation provided, the suggestion is to invoke a random search with the connection object before making the actual operation which is having performance impact. To avoid this, we are tracking the exception and performing 1 retry if timeout happens. Before performing the retry we are invoking pool.releaseDefunctConnection(conn) and getting a fresh connection from the pool. We observed this happened in most of the cases but as we don't know which connection will be pulled from the pool, we are are ending up with LDAPException 85 in some of the retry attempts. Is there a way we can release all the invalid connections before an operation needs to be executed in the connection pool without impacting the performance?
We are using unboundID 6.0.1 version
Last edit: Jyotir 2021-10-13
If you're getting a result code of 85, then that means that the server didn't return a response within the maximum allowed response time. You can set default response timeouts in general and on a per-operation-type basis using the LDAPConnectionOptions class, and you can also set a response timeout on a per-operation basis by using the setResponseTimeoutMillis method on the request. If the problem is that the timeout is legitimately too short for the operation you're processing, then you can use one of those methods for adjusting it.
However, if the problem is that the connection pool has a set of connections that it thinks are valid but actually aren't and can't be used, then there are a couple of main causes for that:
If something like that has happened, then it's unfortunately not possible to passively detect that the connection is no longer valid in a timely manner. The best option that is available to us is TCP keep-alive, and while we do make use of that, it can be hours before the connection is actually considered invalid. As such, the best way of identifying that in a timely manner is to use a combination of health checking and maximum connection age.
The maximum connection age feature allows the LDAP SDK to automatically close and re-establish connections after they've been connected for longer than a specified length of time. That can help prevent them from being seen as "stale" by networking equipment, and it addresses the second class of problems I outlined above.
For the first class of problems, the best way to address that is with health checking. Connection pools can be configured with health checks that can perform a specified set of processing on certain events, including:
The GetEntryLDAPConnectionPoolHealthCheck class provides a simple implementation that tries to retrieve a specified entry, and it will consider the connection invalid if it gets an error while trying to communicate with the server, if the entry doesn't appear to exist or can't be retrieved, or if it takes too long to get a response from the server. If you want to use this or a similar health check, then I'd probably start with just using background health checks for connections not in use, since that doesn't have a significant impact on performance. Invoking a health check on every checkout or release is more expensive because the thread doing the checking out or releasing has to wait on the health check, and the attempt to use the connection for whatever operation you want to process will ultimately do the same basic thing.