UnboundID LDAP SDK for Java

Hello I have this problem where it seems that the LDAP SDK is encoding the values of certain attributes into base64 when they shouldn't be encoded this way. Specifically, the attributes are "displayName", "cn", and "sn". When using the LDAP SDK, it encodes when it saves the value of the attribute and decodes it properly when it gets the attribute back. However, the concern here is storing the attribute value in the LDAP to be used by other applications (including 3rd party applications) that may not recognize the value as base64. Is there any way to force the SDK to NOT encode when saving values in these attributes? Right now, this is how the application performs modifications: List<Modification> modifications = new ArrayList<Modification>(); modifications.add(new Modification(ModificationType.REPLACE, "cn", user.getName().getLastNameOrdered())); modifications.add(new Modification(ModificationType.REPLACE, "sn", user.getName().getLastName())); modifications.add(new Modification(ModificationType.REPLACE, "displayName", user.getName().getFirstNameOrdered())); connection.modify(user.getDN(), modifications); Thanks

Base64 encoding is something that should only be used for the LDIF representation. The actual communication between the client and the server should always use raw binary representations for the values. Similarly, when creating things like attributes or modifications, you should always provide the raw values even if the value should be base64-encoded when represented as LDIF. A quick look at the LDAP SDK code doesn't show any case in which the values are being transformed or altered in any way from what is provided when the modifications were created. Can you give me an example set of values that will demonstrate with the code given above? Neil

Thanks for responding. I figured out that the values the application was storing in these attributes had trailing whitespace, and this was causing the values to be encoded into base64 when output in LDIF. So, for example, while I thought the application was storing a string like "John Smith", it was actually storing a string like "John Smith " (without quotes, only used to show whitespace). Trimming the leading and trailing whitespace from the strings before storing it in these attributes solved the problem, and now they are shown as plaintext when output in LDIF. Thanks for your help.

Thanks for your help Neil, I'll just add that we weren't looking at LDIFF output, but rather at the output from the OpenLdap command line tool. The Directory Server we are using is IBM's Tivoli Directory Server, I don't know if this space padded -> base64 conversion is specific to the IBM LDAP, or if it's standard LDAP, but it sure through us for a loop! Thanks again, Brian Leathem

I'm glad to hear that you figured out what the problem was. Trailing whitespace is a nasty problem. Unfortunately, there may be times in which it is legitimately desired, so it's not safe to simply always remove it. However, the LDIF specification requires that if any value has leading or trailing whitespace, then values MUST be base64-encoded. The corollary to that is that any LDIF data which contains trailing spaces that aren't built into a base64-encoded representation is invalid. In all versions of the LDAP SDK up to and including 2.0.1, the LDIF reader would always include these trailing spaces as part of the value. In 2.1.0 and all subsequent versions, it will reject such entries by default, but you can configure it to automatically strip off these illegal trailing spaces using the LDIFReader.setStripTrailingSpaces method.

And when I say "it's not safe to simply always remove it", I was talking specifically about the LDAP SDK. I don't want to preclude valid use cases in the LDAP SDK. However, it may very well be the case in your environment that you should never expect to have trailing spaces, and in that case it is always safe to remove them in your own application.

I am currently struggling with an application that produces LDIF with trailing spaces and does not encode the value with base64, so I put my related questions in this thread. As you wrote above, the current version of the LDAP SDK throws an Exception if it encounters such entries (or strip the off if the LDIFReader is configured this way). The RFC 2848 states that "Values or distinguished names that end with SPACE SHOULDb e base-64 encoded." (no. 8 of the "Notes on LDIF Syntax"). So I wonder if your interpretation of the RFC is to strict here (you wrote "MUST be base64-encoded" above). Other LDAP tools like the OpenLDAP command line tools or Apache Directory Studio also accept trailing spaces in LDIF files as input (and correctly encode them with base64 in the LDIF output of an LDAP search). I am not sure what the best solution is here, maybe there could be more configuration options for the LDIFReader. And by the way: in the JavaDocs for the "setStripTrailingSpaces" method something about the base64 encoding seems to be missing, the sentence is incomplete: [...] The LDIF specification requires that any value which legitimately contains trailing spaces, and any spaces which appear after the end of values are therefore invalid. [...]

Sorry, there is a typo in my post above: Of course I mean the RFC 2849 (http://tools.ietf.org/html/rfc2849)

You are correct that the LDIF reader should have provided an option to preserve these trailing spaces. To address this, I have just committed a new set of changes that introduce a new TrailingSpaceBehavior enum with values of STRIP, REJECT, and RETAIN, and the LDIF reader now has getTrailingSpaceBehavior and setTrailingSpaceBehavior methods. The previous stripTrailingSpaces and setSkipTrailingSpaces methods are still there, and they have the same effect as before (i.e., if you call setSkipTrailingSpaces(true), then trailing spaces will be silently stripped, and if you call setSkipTrailingSpaces(false), or don't call it at all and get the default behavior, then any entries containing them will be rejected). However, these methods are now deprecated, and it is preferred that you use the methods which deal with TrailingSpaceBehavior values rather than boolean values. Neil

Thanks a lot for your fast response and the nice and flexible solution for dealing with those nasty trailing spaces :-) Now I am facing a second problem with the LDIF export of the same third-party product: it contains a few attributes with duplicated values that are intentionally put there and that I would like to process further. I am not sure if this is permitted by the LDIF RFC, and other LDAP tools treat them differently: the OpenLDAP command line tools keep the duplicated values and the Apache Directoty Studio ignores them in the same way as the LDIFReader. In the LDAP SDK it is possible to create Attributes programmatically that have duplicated values and to add them to a directory. It is also possible to write them to an LDIF file using the LDIFWriter. Only the LDIF reader either ignores them or throws an error, depending on the configuration. I agree that this is the most sensible behavior in most situations, but I wonder if there is an extension point in the LDIFReader where I can deal with the input differently in this special case. Mathis

I've just committed another change which introduces a DuplicateValueBehavior enum with values of IGNORE, REJECT, and RETAIN. It works in much the same way as the earlier TrailingSpaceBehavior enum. The default behavior has not changed, and the existing ignoreDuplicateValues and setIgnoreDuplicateValues methods are still there (although now they're deprecated in favor of getDuplicateValueBehavior and setDuplicateValueBehavior). Neil

I've just slightly modified my latest commit to rename the IGNORE option to be STRIP. The word "strip" is clearer in its action, and it's more consistent with the values of the TrailingSpaceBehavior enum.

And thanks again! Its great that you provided some flexible solutions, even though I am currently trying to deal with some ugly edge cases. This time I also build the LDAP SDK from the SVN for the first time, and it worked quite nice as well. Only Checkstyle complained about some missing new lines at the end of some Java files. Maybe the default settings for Checkstyle use the wrong line separator when building under Windows. Mathis

Thanks for pointing out the Windows build problem. For some reason, my Subversion workspace was lying to me and told me that several files had the svn:eol-style property set when they really didn't. When I did a fresh checkout and checked again, I could see several files missing that property. I've corrected the problem, and it should build just fine now. Sorry for the inconvenience. Neil