Updated the round-robin and fewest connections server sets so that they can temporarily blacklist a server that was found to be offline or unavailable. If an attempt to create a connection to a server fails, or if that connection is found to be unacceptable for some reason (e.g., it does not pass the associated health check), subsequent connection attempts will avoid that server until a background thread determines that it is available again. Blacklisted servers will still be tried as a last resort if it is not possible to get an acceptable connection to a non-blacklisted server. These server sets will now use the blacklist by default, but that can be disabled programmatically through the constructor or by setting a system property before creating the server set.
Updated the round-robin and fewest connections server sets to improve concurrency. In previous implementations, these sets could only create one connection at a time, which could limit the rate at which connection pools using them could establish new connections. This is no longer the case, and any number of threads will be able to create connections in parallel using the server sets. This change also updated the ServerSet API to make it possible for a server set to be notified whenever a connection created with that set has been closed.
Added a new SubtreeDeleter utility class that can make it easier to delete a specified subtree, optionally including or excluding the base entry for that subtree. It provides a good client-side alternative to the subtree delete request control, which isn’t supported by all servers and can sometimes be problematic in servers that do support it.
Added a new ldapdelete command-line tool that can be used to delete entries from an LDAP directory server. The DNs of the entries to delete can be provided on the command line, read from a file, or read from standard input. Alternately, the server can search for and delete all entries matching one or more filters. It offers a number of options, including support for client-side and server-side subtree deletes, rate limiting, and a variety of standard and proprietary controls.
Improved the LDAP SDK’s protection against socket write attempts that block for an indefinite length of time. This is only likely to occur when sending a large number of asynchronous requests over a connection, and only in the case that the server stops reading requests from the client or if a networking problem prevents the request from reaching the server and prevents the client from receiving any information about that failure.
Added InMemoryDirectoryServer.applyChangesFromLDIF methods that can be used to read LDIF change records and apply them to data in the server. The changes will be applied atomically, and if any of them cannot be applied successfully, then the server data will remain unchanged.
Updated the searchrate utility to allow specifying the base DN, scope, filter, and requested attributes using LDAP URLs rather than using separate arguments to provide appropriate values. The LDAP URL can be a fixed URL, or it can be a value pattern (including the ability to include variable content in the URLs or to load the URLs from a file). Using LDAP URLs allows for more precise control over the combination of base, scope, filter, and requested attributes on a per-request basis. Note that any addresses and ports used in the URLs will be ignored; the --hostname and --port arguments will still be used to identify which servers to use.
Updated the ldapsearch and ldapmodify command-line tools to use an unlimited response timeout, which will prevent the tool from giving up on an operation if it takes the server a long time to return any kind of response. Previously, the tools used the LDAP SDK’s default timeout of five minutes for searches and 30 seconds for add, delete, modify, and modify DN operations.
Updated the ldapmodify command-line tool to add a --clientSideSubtreeDleete argument that can be used to cause each delete operation to be converted to a client-side subtree delete operation, in which the tool will search for entries to delete and then delete them individually. This makes it easier to delete entries with subordinates on servers that either do not support the subtree delete request control or in which the client may not have permission to use that control.
Added a new indent-ldap-filter command-line tool that can help make it easier to visualize complex filters with a lot of components, and especially a lot of nesting. If possible, it can also try to simplify the filter (for example, to remove unnecessary levels of nesting, like an AND inside an AND).
Enabled concurrent socket factory use by default for all versions of Java. In the past, we have observed that at least some IBM JVMs had a thread safety issue with SSL socket factory implementations, so we only allowed a socket factory to be used concurrently by multiple threads on a whitelisted set of JVMs. We no longer believe that the IBM JDK socket factory thread safety is an issue, and there are now many more JVM vendors (e.g., Apple, Azul, Amazon Coretto, AdoptOpenJDK, and potentially Red Hat), so concurrent socket factory use will be enabled by default. If an issue is found on a particular JVM, then concurrent access can be disabled programmatically or with a system property.
Updated the LDAPCommandLineTool API to add an option to expose an --enableSSLDebugging argument. If this argument is available, and if it is provided in the set of command-line arguments when the tool is run, then the JVM’s SSL/TLS debugging support will be enabled, and the JVM will write a large amount of TLS-related debugging information to standard error. This can help troubleshoot problems with or provide detailed information about any TLS communication that the tool attempts.
Updated the LDAP SDK to add protection against JVM security managers that may prevent calls to certain methods, like attempts to interact with system properties, environment variables, or logger levels.
Updated the password reader so that it will generate a more user-friendly error message if it is run in a context in which no console is available. A tool could encounter this error if its output has been redirected, or if it’s not running in an interactive shell (for example, in a cron job or system startup script).
Dramatically improved the performance of the streamfile value pattern, which operates like the sequentialfile value pattern in that it can iterate through values in sequential order, except that streamfile doesn’t need to hold the whole file in memory at once whereas sequentialfile does.
Updated the Filter.simplifyFilter method to simplify an AND filter containing an LDAP false filter (an OR filter with zero components, which will never match anything) to just that LDAP false filter, and to simplify an OR filter containing an LDAP true filter (an AND filter with zero components, which will match any entry) to just that LDAP true filter.
Added a PasswordValidationDetailsResponseControl.get(LDAPException) method that makes it more convenient to get the response control from an unsuccessful operation.
Improved the exception message that is generated if a failure occurs while trying to create a TLS-based connection. If the JVM supports creating an unconnected SSLSocket and then connecting it after the fact (which makes it possible to specify a connect timeout), and that connection attempt failed (for example, because the client did not trust the certificate presented by the server), the LDAP SDK could think that the connection was still established. Subsequent attempts to use the connection would fail, but the failure message would not accurately reflect the true cause of the problem.
Updated the in-memory directory server to improve the diagnostic message that is returned when it rejects an add attempt because the provided entry is not within any of the configured base DNs.
Fixed an issue in generating the normalized representation of a multivalued RDN when one or more of those components referenced an attribute type by its OID or by a name other than the first one listed in the attribute type definition. Previously, the normalized string representation would have simply used an all-lowercase representation of the provided attribute name, but it will now use an all-lowercase representation of the primary name for that attribute (if schema information is available to the client). Also, updated the logic used to determine whether an RDN has a specified name or name-value pair to handle the use of alternate names, and exposed the RDN.getNameValuePairs method to make it easier to work with an RDN’s name-value pairs.
Fixed a bug in the ByteStringBuffer.append(CharSequence,int,int) method in which the final integer argument could be interpreted as the number of characters to append rather than the end position at which to stop appending, which could yield incorrect results when the method was called with a nonzero start position. Also, updated the ByteStringBuffer.append methods that take CharSequence arguments to eliminate the creation of an intermediate character array, thereby improving performance and reducing garbage creation.
Updated the LDAP SDK’s command-line tool framework to fix an issue with the tool’s validation for required, exclusive, and dependent argument sets. If an argument was configured with a default value, then that default value could have been mistakenly treated as if it had been explicitly provided by the user. This could cause problems for arguments that are part of an exclusive argument set (in which only one of the arguments in that set may be provided) or a dependent argument set (in which an argument can only be used if at least one of a specified set of additional arguments is present). In such cases, the tool could not have been used in interactive mode. The modrate tool was affected by this issue.
Updated the argument parser to fix a problem with the way that it handles backslash characters in argument property files. Previously, it only correctly handled backslashes if they were at the end of a line to indicate that the content continued to the next line, or if they were followed by the letter ‘u’ and the hexadecimal representation of the desired Unicode character. It did not handle the backslash in front of another character used to force that character to be treated as a literal (for example, a backslash followed by an equal sign should be treated as just an equal sign, but was instead being treated as a backslash followed by an equal sign).
Updated the LDAP SDK build process to automatically generate documentation for all of the command-line tools included with the LDAP SDK. Usages are provided in both plain text and HTML, and the HTML-formatted documentation is linked from the main documentation index page.
Last edit: Neil Wilson 2019-06-04
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
We have just released version 4.0.11 of the UnboundID LDAP SDK for Java. It is available for download from the releases page of our GitHub repository (https://github.com/pingidentity/ldapsdk/releases), from the Files page of our SourceForge repository (https://sourceforge.net/projects/ldap-sdk/files/), and from the Maven Central Repository (https://search.maven.org/search?q=g:com.unboundid%20AND%20a:unboundid-ldapsdk&core=gav).
The LDAP SDK release notes are available at https://docs.ldap.com/ldap-sdk/docs/release-notes.html, but the changes included in this release are as follows:
Updated the round-robin and fewest connections server sets so that they can temporarily blacklist a server that was found to be offline or unavailable. If an attempt to create a connection to a server fails, or if that connection is found to be unacceptable for some reason (e.g., it does not pass the associated health check), subsequent connection attempts will avoid that server until a background thread determines that it is available again. Blacklisted servers will still be tried as a last resort if it is not possible to get an acceptable connection to a non-blacklisted server. These server sets will now use the blacklist by default, but that can be disabled programmatically through the constructor or by setting a system property before creating the server set.
Updated the round-robin and fewest connections server sets to improve concurrency. In previous implementations, these sets could only create one connection at a time, which could limit the rate at which connection pools using them could establish new connections. This is no longer the case, and any number of threads will be able to create connections in parallel using the server sets. This change also updated the ServerSet API to make it possible for a server set to be notified whenever a connection created with that set has been closed.
Added a new
SubtreeDeleterutility class that can make it easier to delete a specified subtree, optionally including or excluding the base entry for that subtree. It provides a good client-side alternative to the subtree delete request control, which isn’t supported by all servers and can sometimes be problematic in servers that do support it.Added a new
ldapdeletecommand-line tool that can be used to delete entries from an LDAP directory server. The DNs of the entries to delete can be provided on the command line, read from a file, or read from standard input. Alternately, the server can search for and delete all entries matching one or more filters. It offers a number of options, including support for client-side and server-side subtree deletes, rate limiting, and a variety of standard and proprietary controls.Improved the LDAP SDK’s protection against socket write attempts that block for an indefinite length of time. This is only likely to occur when sending a large number of asynchronous requests over a connection, and only in the case that the server stops reading requests from the client or if a networking problem prevents the request from reaching the server and prevents the client from receiving any information about that failure.
Added
InMemoryDirectoryServer.applyChangesFromLDIFmethods that can be used to read LDIF change records and apply them to data in the server. The changes will be applied atomically, and if any of them cannot be applied successfully, then the server data will remain unchanged.Updated the
searchrateutility to allow specifying the base DN, scope, filter, and requested attributes using LDAP URLs rather than using separate arguments to provide appropriate values. The LDAP URL can be a fixed URL, or it can be a value pattern (including the ability to include variable content in the URLs or to load the URLs from a file). Using LDAP URLs allows for more precise control over the combination of base, scope, filter, and requested attributes on a per-request basis. Note that any addresses and ports used in the URLs will be ignored; the--hostnameand--portarguments will still be used to identify which servers to use.Updated the
ldapsearchandldapmodifycommand-line tools to use an unlimited response timeout, which will prevent the tool from giving up on an operation if it takes the server a long time to return any kind of response. Previously, the tools used the LDAP SDK’s default timeout of five minutes for searches and 30 seconds for add, delete, modify, and modify DN operations.Updated the
ldapmodifycommand-line tool to add a--clientSideSubtreeDleeteargument that can be used to cause each delete operation to be converted to a client-side subtree delete operation, in which the tool will search for entries to delete and then delete them individually. This makes it easier to delete entries with subordinates on servers that either do not support the subtree delete request control or in which the client may not have permission to use that control.Added a new
indent-ldap-filtercommand-line tool that can help make it easier to visualize complex filters with a lot of components, and especially a lot of nesting. If possible, it can also try to simplify the filter (for example, to remove unnecessary levels of nesting, like an AND inside an AND).Enabled concurrent socket factory use by default for all versions of Java. In the past, we have observed that at least some IBM JVMs had a thread safety issue with SSL socket factory implementations, so we only allowed a socket factory to be used concurrently by multiple threads on a whitelisted set of JVMs. We no longer believe that the IBM JDK socket factory thread safety is an issue, and there are now many more JVM vendors (e.g., Apple, Azul, Amazon Coretto, AdoptOpenJDK, and potentially Red Hat), so concurrent socket factory use will be enabled by default. If an issue is found on a particular JVM, then concurrent access can be disabled programmatically or with a system property.
Updated the
LDAPCommandLineToolAPI to add an option to expose an--enableSSLDebuggingargument. If this argument is available, and if it is provided in the set of command-line arguments when the tool is run, then the JVM’s SSL/TLS debugging support will be enabled, and the JVM will write a large amount of TLS-related debugging information to standard error. This can help troubleshoot problems with or provide detailed information about any TLS communication that the tool attempts.Updated the LDAP SDK to add protection against JVM security managers that may prevent calls to certain methods, like attempts to interact with system properties, environment variables, or logger levels.
Updated the password reader so that it will generate a more user-friendly error message if it is run in a context in which no console is available. A tool could encounter this error if its output has been redirected, or if it’s not running in an interactive shell (for example, in a cron job or system startup script).
Dramatically improved the performance of the
streamfilevalue pattern, which operates like thesequentialfilevalue pattern in that it can iterate through values in sequential order, except thatstreamfiledoesn’t need to hold the whole file in memory at once whereassequentialfiledoes.Updated the
Filter.simplifyFiltermethod to simplify an AND filter containing an LDAP false filter (an OR filter with zero components, which will never match anything) to just that LDAP false filter, and to simplify an OR filter containing an LDAP true filter (an AND filter with zero components, which will match any entry) to just that LDAP true filter.Added a
PasswordValidationDetailsResponseControl.get(LDAPException)method that makes it more convenient to get the response control from an unsuccessful operation.Improved the exception message that is generated if a failure occurs while trying to create a TLS-based connection. If the JVM supports creating an unconnected
SSLSocketand then connecting it after the fact (which makes it possible to specify a connect timeout), and that connection attempt failed (for example, because the client did not trust the certificate presented by the server), the LDAP SDK could think that the connection was still established. Subsequent attempts to use the connection would fail, but the failure message would not accurately reflect the true cause of the problem.Updated the in-memory directory server to improve the diagnostic message that is returned when it rejects an add attempt because the provided entry is not within any of the configured base DNs.
Fixed an issue in generating the normalized representation of a multivalued RDN when one or more of those components referenced an attribute type by its OID or by a name other than the first one listed in the attribute type definition. Previously, the normalized string representation would have simply used an all-lowercase representation of the provided attribute name, but it will now use an all-lowercase representation of the primary name for that attribute (if schema information is available to the client). Also, updated the logic used to determine whether an RDN has a specified name or name-value pair to handle the use of alternate names, and exposed the
RDN.getNameValuePairsmethod to make it easier to work with an RDN’s name-value pairs.Fixed a bug in the
ByteStringBuffer.append(CharSequence,int,int)method in which the final integer argument could be interpreted as the number of characters to append rather than the end position at which to stop appending, which could yield incorrect results when the method was called with a nonzero start position. Also, updated theByteStringBuffer.appendmethods that takeCharSequencearguments to eliminate the creation of an intermediate character array, thereby improving performance and reducing garbage creation.Updated the LDAP SDK’s command-line tool framework to fix an issue with the tool’s validation for required, exclusive, and dependent argument sets. If an argument was configured with a default value, then that default value could have been mistakenly treated as if it had been explicitly provided by the user. This could cause problems for arguments that are part of an exclusive argument set (in which only one of the arguments in that set may be provided) or a dependent argument set (in which an argument can only be used if at least one of a specified set of additional arguments is present). In such cases, the tool could not have been used in interactive mode. The
modratetool was affected by this issue.Updated the argument parser to fix a problem with the way that it handles backslash characters in argument property files. Previously, it only correctly handled backslashes if they were at the end of a line to indicate that the content continued to the next line, or if they were followed by the letter ‘u’ and the hexadecimal representation of the desired Unicode character. It did not handle the backslash in front of another character used to force that character to be treated as a literal (for example, a backslash followed by an equal sign should be treated as just an equal sign, but was instead being treated as a backslash followed by an equal sign).
Updated the LDAP SDK build process to automatically generate documentation for all of the command-line tools included with the LDAP SDK. Usages are provided in both plain text and HTML, and the HTML-formatted documentation is linked from the main documentation index page.
Last edit: Neil Wilson 2019-06-04