Add filtering ability to CsvToBean
Brought to you by:
aruckerjones,
sconway
Menu
▾
▴
1 Attachments
#67 Add filtering ability to CsvToBean
I'm going to be parsing some very large CSV files (hundreds of megabytes). I don't want to keep all the data. If I let CsvToBean process the entire file before I throw away the parts I don't want, it would consume a lot of memory. I'd like CsvToBean to filter the data while it's being read from the file.
I attached a patch which adds the functionality I want.
Oops. The patch I included when I opened the ticket is missing a couple of files. Here's the complete patch.
I like it but I am hesitant at the same time.
The issue I see with this is that the user has to know that the code that they are testing/filtering is the preprocessed data. So lets say that you have created a data file that has the quote and escape char as single quote ( ' ) and you have a column businessName and the value is Al's
In the data file I would expect to see 'Al''s' and in the object would be Al's
If I was writing a filter to remove all the Al's I would have to know to search for Al''s
Talking it through it is a logical assumption but just needs to be documented very well. Let me run this by our other, more active, developer and see what he thinks.
Sorry this has been almost a year but work and family have been real busy for the last several years and this is a small project with few developers so at any one time there is usually no one active on the board.
Scott :)
merged into trunk and slated for 3.1 version.