LDAPConnectionPool Health Check Connection release issue

[RPC]

Hi Neil,

I have created the LDAPReadWriteConnectionPool with the below configuration
LDAPReadConnectionPool (Min - 3, Max - 5) - Assigned one instance of HeathCheck class
LDAPWriteConnectionPool (Min - 2, Max - 5) - Assigned one instance of HealthCheck class
HealthCheck Interval time - 30 Seconds
HealthCheck Idle time - 1 Minute

Created the customized class which overrides the LDAPConnectionPoolHealthCheck and overrides the method like below

 @Override
    public void ensureConnectionValidForContinuedUse(LDAPConnection connection)
            throws LDAPException
    {
        Long idleTime = (System.currentTimeMillis() - connection.getLastCommunicationTime());
        if (idleTime > connectionIdleTimeout) {
            ldapConnectionPool.releaseConnection(connection);
            LOGGER.debug("Method ensureConnectionValidForContinuedUse - Connectionpool conenction released "+ "+ ldapConnectionPool.getConnectionPoolName());
        }
    }

Kindly check the log below
12:44:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released WRITE POOL
12:44:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released READ POOL
12:44:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released WRITE POOL
12:44:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released READ POOL

12:44:45 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released READ POOL
12:44:45 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released WRITE POOL
12:44:45 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released READ POOL
12:44:45 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released WRITE POOL
12:44:45 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released READ POOL

12:45:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released WRITE POOL
12:45:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released WRITE POOL
12:45:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released READ POOL
12:45:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released READ POOL
12:45:45 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released WRITE POOL

12:45:45 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released WRITE POOL
12:45:45 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released READ POOL
12:45:45 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released READ POOL

12:46:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released WRITE POOL
12:46:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released WRITE POOL
12:46:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released READ POOL
12:46:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released READ POOL
12:46:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released READ POOL
12:46:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released READ POOL
12:46:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released READ POOL
12:46:15 Method ensureConnectionValidForContinuedUse - Connectionpool conenction released WRITE POOL

It prints weirdly. For few minutes it is printing properly as expected. After it is printing lot of message. For Ex: I have created the LDAPReadConnectionPool with min 3 size and I have performed only one read operation. After that, I have not performed any operations (Both read and write). But as per the logs,there are 5 Connections of Read Pool gets released, later count increased to 6, 7 ..... for Read Connection Pool. Same behavior for LDAPWriteConnectionPool.

I have changed the code from release connection to discard connection, there too same issue persist.

Later I have changed the code, by checking the connection availability (connection.isConnected()), then try to release the connection, same issue.

Kindly suggest the way to properly release the connection with the time interval for the idle connections.If you share some working sample it would be great for us to fix the issue.

Thanks for your help in advance.

[Neil Wilson]

You should definitely not release a connection back to the connection pool in a health check. If a connection is fine and can continue to be used then just release it without throwing any exception. If a connection is no longer valid, then throw an exception and the connection pool will close that connection and create a new one to take its place.

If you explicitly call releaseConnection like you're doing here, then you'll end up with the same connection in the pool twice and that's a very bad thing because the LDAP SDK expects to have exclusive access to a connection when it's in the pool and this could mean that the same connection object could be both checked out and checked in simultaneously.

Similarly, if you explicitly call releaseDefunctConnection from inside a health check instead of throwing an exception, then you'll end up causing both a newly-created connection to be added to the pool and the connection that got closed will also be put in the pool and will throw an exception the next time you try to use it. A similar result will occur if you explicitly call discardConnection.

There really isn't any way to completely get rid of a specific connection in an LDAP health check without having a new connection created to take its place. Health checks aren't designed for that purpose. The only way you could get rid of connections that have been idle for too long without having them be replaced with newly-created connections would be to have your own thread that periodically checks connections out of the pool, sees if they've been idle for too long, and if so then calls discardConnection instead of releaseConnection. But you cannot do that in a health check because it will cause bad things to happen.

[RPC]

Thanks Neil for the detailed response.

If there are 5 minimum connections, 10 maximum connections in the LDAPWriteConnectionPool.
At the max System requires only 2 connections for certain period. Now I would like to shrink the WriteConnectionPool size to 2, because other 3 connections are idle for long time & consuming the resources.

Question 1. Do we have any method to shrink the pool using Health Check (If the connection is idle for long time)?
Question 2. If yes for Question 1, If system needs 8 connections at certain point of time, will it create the connections in the Write Pool?

[Neil Wilson]

Answer 1: There is no supported way to shrink the size of a connection pool in a health check. If you want to close a connection that has been idle for too long, you can throw an exception in the health check's ensureConnectionValidForContinuedUse method, but that will result in a new connection being established to take its place. If you want to reduce the size of the pool, you need to do it outside of a health check.

Answer 2: By default, if the connection pool needs a connection and none are currently available (e.g., because they're all checked out, or because there was a directory server outage), then the connection pool will create a new connection to use. If you're using a read-write connection pool, then it actually maintains two different connection pools behind the scenes, and the new connection will be created in the pool that is appropriate for the type of operation you're trying to process.

[RPC]

Thanks Neil.
Up to my understanding
- If I would like to shrink, then I have to create the connection pool again with the new min & max size. If there is any other option, kindly share the way to proceed (Code example).
- If we close 2 connections in the Connection pool out of 5 (Max). Later, if it requires more connection, then Connection pool takes care of creating the new connections up to the maximum configured limit.

[Neil Wilson]

First, let me recommend against shrinking the size of a connection pool. Most of the time, it just doesn't make sense. Establishing a connection can be relatively expensive, especially if there's a security layer like SSL/TLS involved. If you need a connection and there isn't one available, then needing to establish a new one before you can process an operation can dramatically increase the time required to process that operation. On the other hand, the cost of keeping an already-established connection is negligible.

I understand that some unfortunately common network hardware has a bad habit of cutting off connections after a period of time (and often do so in a way that is completely undetectible to either end of the connection) and that some directory servers may close connections after some period of time (but at least the good ones tell the client about it before they do it). To handle these situations, the LDAP SDK offers a maximum connection age feature that allows the connections to be pre-emptively closed and replaced with newly-created connections before the timeout is encountered.

If you create the connection pool with a small initial size and it grows beyond that, then it means that there was heavy enough use to require them at least once, so it's not unreasonable to assume that they might be needed again. And it's better to already have them available than to have to create them on demand. But the default behavior for the connection pool (which you can change if desired) is to create a connection if one is required but there aren't any immediately available.

However, if you think that there is a legitimate case for shrinking the pool, then the LDAP SDK provides a discardConnection method that can be used to close a connection without creating a new one to take its place (and thereby shrinking the pool by one connection). You should never call this method from inside a health check (just as you should never call releaseConnection or releaseDefunctConnection), but you can call it from any of your own application threads. The next release of the LDAP SDK will include a shrinkPool method that can automate the process of checking out one or more connections and discarding them to get the pool down to a specified size, but it doesn't do anything that isn't already available.