Modifying connection option for already connected connection

[www.pakistanandroid.com]

Hi,

I have created a connectionpool from which i get a connection.

The connection calls a search method and timesout after predefined 10 seconds and an exception is thrown.

In the exception we modify the connection option to set setResponseTimeoutMillis to 0 (unlimited timeout i.e. blocking connection till we get response from server).

Is this ok? Because In api documentation its said for setConnectionOptions that :

setConnectionOptions:

Specifies the set of connection options for this connection. Some changes may not take effect for operations already in progress, and some changes may not take effect for a connection that is already established.

Now I wonder will the setResponseTimeoutMillis change will take effect for already established connection?

Thanks

BR
Shahzad

[Neil Wilson]

There are a number of options that may not take effect for operations that are already in progress on the connection, and the response timeout is one of them. However, changes to the response timeout should take effect for any new operations requested after the connection options are updated for the connection.

Note that you can also set the response timeout for individual operations through the setResponseTimeoutMillis method on the request (e.g., SearchRequest.setResponseTimeoutMillis). But regardless of the method you use to set the response timeout, I would strongly caution against setting an unlimited timeout for a request in an application intended to run in a production environment. If you set an unlimited request timeout, then if the server never sends a response for that operation (e.g., if the operation is aborted by an administrator or if the server encounters an internal error that causes it to abandon processing for that operation without sending a response) then the thread processing the request may be blocked forever. It's generally better to use a really long timeout than no timeout at all.

In general, changes that won't take effect for an already-established connection are those that affect how a connection should be established. They include:

• captureConnectStackTrace
• connectTimeout
• useKeepAlive
• useLinger
• lingerTimeout
• useReuseAddress
• useSchema
• usePooledSchema
• useSynchronousMode
• useTCPNoDelay
• referralConnector
• receiveBufferSize
• sendBufferSize

It is worth noting that while the referral connector referenced in the connection options will only be looked at when the connection is established, you can change it for an already-established connection using the LDAPConnection.setReferralConnector method. That's kind of inconsistent with the rest of the API, but changing it would break backward compatibility so it's going to stay the way it is.

Neil

[www.pakistanandroid.com]

Thank you very much Neil. Very helpful. I really appreciate your efforts for answering my queries. Thank you :)

[www.pakistanandroid.com]

Hi Neil,

One important question which I forgot to mention in my last reply.

I am using LDAPConnectionPool to get LDAP connection. I am using search method to search for records which I get in SearchResult data type.

Now its fine with current implementation to retrieve couple of thousand of records. We have set timeout of 10 seconds to retrieve those record and it works very well.

But we are going to a situation when we will have millions of records to retrieve in SearchResult data type.

• Do you think using search method to search for millions of records and retreive them to SearchResult data type is ok? (using one connection from pool and synchronous)
• Do you think we might need to set some timeout of 30 minutes or may be an hour to retrieve all those records?
• Do you have some better way of doing it. I mean some good practices for retrieving millions of records.

I hope my questions would be clear to you or else I can explain them more. Please let me know.

Thank you

BR
Shahzad

[Neil Wilson]

If you want to retrieve a large number of entries, you should definitely not use the default search method, since it collects all the results in a list, which can consume a lot of memory. Also, the server may be configured to limit the maximum number of entries that should be returned from any search.

The recommended approach for a search that may return a large number of entries would be to use the simple paged results control, which allows you to perform a search that returns no more than a specified number of entries, but also includes a token that you can use to resume the search and get the next batch of entries. The UnboundID LDAP SDK supports this control in the com.unboundid.ldap.sdk.controls.SimplePagedResultsControl class, and the javadoc documentation for that class includes an example that demonstrates how to use it.

Note that if the directory server you are using doesn't support the simple paged results control, you can check to see if it supports the virtual list view (VLV) control instead. This control provides you more options than the simple paged results control, but it's also more expensive to use and some servers only allow you to use it with special indexes defined.

If the server you're using doesn't support either of these controls, then you can try to get all the results in a single search, but in that case you should use a SearchResultListener to process each entry as it's returned rather than having them all collected in a list that is included in the SearchResult object.

Neil

[www.pakistanandroid.com]

Hi Neil,

Thank you very much. We are trying to figure out our LDAP server configuration and its support of your mentioned SimplePagedResultsControl and VLV and will do the implementation accordingly.

About LDAPConnectionPool, when we call getconnection method which gives us a connection from pool, does it mean that the connection is already connected to the server or is it the search method which initiates connection to the server and perform search and returns the result.

Thank you

BR
Shahzad
Thank you

[Neil Wilson]

When you create a connection pool, it will establish the specified initial number of connections, so it should be ready to use right away. However, if there aren't any connections available for some reason (e.g., they're all already in use processing operations, or there was a problem with the server that caused the existing connections to be closed), then the connection pool can create connections on the fly.

Also note that while you certainly can check connections out of the pool, that probably isn't necessary most of the time. Instead of using:

 LDAPConnection conn = pool.getConnection();
 SearchResult searchResult = conn.search(searchRequest);
 pool.releaseConnection(conn);

you can instead just use:

 SearchResult searchResult = pool.search(searchRequest);

Not only is this easier and more convenient, but it's also safer because the connection pool does all the work of checking out the connection and making sure it gets checked back into the pool after the operation is done, or in the event of a problem with the connection, ensures that it is destroyed and a new connection gets established and added to the pool in its place.

Neil

[www.pakistanandroid.com]

Thank you Neil. I will take this in to practice.

Thanks once again for help. :)