In ByteArrayOutputStream not been flushed problem, I change the PrintWriter constructor to include the autoFlush flag and it solved the problem. But I still do not quite understand the reason. In the finally clause, I do have writer.close() statement and why WriteAttrToFileUsingListener.java does not need to force to flush the buffer ?
Second, In finding the solution to this problem, I rewrote the searchEntryReturned method. I did not define the PrintWriter variable in the class, instead I defined a Vector variable in the class, so in searchEntryReturned method, instead of using PrintWriter to write the entry to an output stream, the entry was added to the vector and a getVector method to return the Enumeration of this vector. My question is that is this a good solution? or write entry to output stream is better?
Thank you
Yi
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm not entirely sure why just calling close didn't flush the stream. It's probably a quirk in the way that PrintStream and ByteArrayOutputStream interact with each other.
With regard to your question about using a Vector to hold the results, I have a couple of comments. First, I would never recommend the use of Vector under any circumstances. Vector is a really old class that has unnecessary synchronization. It's almost always better to use ArrayList or LinkedList, and if you need synchronization then do it manually because what Vector provides probably isn't what you want.
Second, if you want all of your results collected in a list, then there's no need to use a SearchResultListener at all. If you don't use a SearchResultListener, then the default behavior is to collect the entries in a list, and you can get to them using the SearchResult.getSearchEntries() method. This is provided as a convenience, but the potential problem is that if you have a very large result set, then holding them all at once could require more memory than you have on the client. As a result, if you have a very large result set, then it's better to deal with the entries individually using either a SearchResultListener, or iterate across them using an LDAPEntrySource.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have an implemented class using LDAPEntrySource, the only problem is if I go through the entire LDAP server records, it requires lots of memory. Please correct me if I was wrong - it loads all the attributes of all records into RAM. Even I assigned all the 1G RAM in VM to it, still not enough, what I need is just one attribute. So SearchResultListener solved this problem by using much less RAM.
Regarding to Vector usage, I used ArrayList at the beginning, but JVM threw out of heap space error. So I changed to Vector and JVM stopped complain. I posted my question about why Vector does not cause out of heap space error in this case on JAVA forum, someone bumped me with very mean answer.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It doesn't make sense that using an LDAPEntrySource by itself would require a significant amount of memory. By default, it will only hold 100 entries in memory at any given time, but that limit is configurable at creation time. If the search matches more entries than that and the queue that the LDAPEntrySource uses to hold them until they can be processed gets full, then the LDAP SDK will temporarily stop reading data from the server on that connection until space has been freed up in the queue.
Also, because you create the LDAPEntrySource with a SearchRequest object, the entries that are returned will only create those attributes which were named in the search request (unless you didn't specify any requested attributes, in which case the default behavior is to request all user attributes).
If you are running out of memory when using the LDAPEntrySource, then it's likely something that you were doing with the entries you get from the LDAPEntrySource (e.g., keeping a reference to them somewhere else that prevents them from being garbage collected and freed).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for the help. The out of memory is not caused by using LDAPEntrySource. But one thing I noticed using SearchResultListener is much faster for same query. Although, I did not do exact speed testing.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
In ByteArrayOutputStream not been flushed problem, I change the PrintWriter constructor to include the autoFlush flag and it solved the problem. But I still do not quite understand the reason. In the finally clause, I do have writer.close() statement and why WriteAttrToFileUsingListener.java does not need to force to flush the buffer ?
Second, In finding the solution to this problem, I rewrote the searchEntryReturned method. I did not define the PrintWriter variable in the class, instead I defined a Vector variable in the class, so in searchEntryReturned method, instead of using PrintWriter to write the entry to an output stream, the entry was added to the vector and a getVector method to return the Enumeration of this vector. My question is that is this a good solution? or write entry to output stream is better?
Thank you
Yi
I'm not entirely sure why just calling close didn't flush the stream. It's probably a quirk in the way that PrintStream and ByteArrayOutputStream interact with each other.
With regard to your question about using a Vector to hold the results, I have a couple of comments. First, I would never recommend the use of Vector under any circumstances. Vector is a really old class that has unnecessary synchronization. It's almost always better to use ArrayList or LinkedList, and if you need synchronization then do it manually because what Vector provides probably isn't what you want.
Second, if you want all of your results collected in a list, then there's no need to use a SearchResultListener at all. If you don't use a SearchResultListener, then the default behavior is to collect the entries in a list, and you can get to them using the SearchResult.getSearchEntries() method. This is provided as a convenience, but the potential problem is that if you have a very large result set, then holding them all at once could require more memory than you have on the client. As a result, if you have a very large result set, then it's better to deal with the entries individually using either a SearchResultListener, or iterate across them using an LDAPEntrySource.
Thanks for the your reply.
I have an implemented class using LDAPEntrySource, the only problem is if I go through the entire LDAP server records, it requires lots of memory. Please correct me if I was wrong - it loads all the attributes of all records into RAM. Even I assigned all the 1G RAM in VM to it, still not enough, what I need is just one attribute. So SearchResultListener solved this problem by using much less RAM.
Regarding to Vector usage, I used ArrayList at the beginning, but JVM threw out of heap space error. So I changed to Vector and JVM stopped complain. I posted my question about why Vector does not cause out of heap space error in this case on JAVA forum, someone bumped me with very mean answer.
It doesn't make sense that using an LDAPEntrySource by itself would require a significant amount of memory. By default, it will only hold 100 entries in memory at any given time, but that limit is configurable at creation time. If the search matches more entries than that and the queue that the LDAPEntrySource uses to hold them until they can be processed gets full, then the LDAP SDK will temporarily stop reading data from the server on that connection until space has been freed up in the queue.
Also, because you create the LDAPEntrySource with a SearchRequest object, the entries that are returned will only create those attributes which were named in the search request (unless you didn't specify any requested attributes, in which case the default behavior is to request all user attributes).
If you are running out of memory when using the LDAPEntrySource, then it's likely something that you were doing with the entries you get from the LDAPEntrySource (e.g., keeping a reference to them somewhere else that prevents them from being garbage collected and freed).
Thanks for the help. The out of memory is not caused by using LDAPEntrySource. But one thing I noticed using SearchResultListener is much faster for same query. Although, I did not do exact speed testing.