Re: [dbunit-developer] questions regarding data exchange

[Jeff J. <jj...@ap...>]

On Fri, Dec 17, 2010 at 2:02 PM, Albrecht Gass <ag...@ct...> wrote:
> Hi,
>
> we are trying to us dbunit to extract data from a multi-lingual database
> from an installation and import the data on developers machines for their
> own testing. I have found the basics of dbunit to be working well but have a
> few questions/concerns:
>
> 1) What is the best file format to use to exchange mutli-lingual data? I
> investigated FlatXml* classes but since XML imposes limitations on the valid
> characters that can be stored in XML (characters less than 0x20 are
> considered invalid) this is not an option. The Csv* classes seem to not have
> this limitation.
> Assuming CSV is the proper format for the data exchange, this leads to more
> questions after further investigation (based on 2.4.8):

I am not familiar with this, sorry...  hopefully someone else is and will reply.


> 2) CsvDataSetWriter, CsvDataSet use the default character encoding when
> writing and reading of the CSV files. This seems to limit the data to
> western characters. Should the CSV files always be written in UTF-8 or
> should the character set be configurable.

Sounds like a limitation of the original design.  If you want to patch it...


> 3) Why is there an interface CsvParser defined but no alternative
> implementation to CsvParserImpl can be used in CsvProducer? Should it be
> possible to create an alternative CSV parser and set it on the CsvProducer
> instance?

Sounds like a limitation of the original design.  If you want to patch it...


> 4) The CsvParserImpl seems to have severe performance issues with large
> files (we have a file that contains ~7000 records and a size of 200MB.
> CsvParserImpl seems to never finish parsing the file. I have written an
> alternative parser that can parse the file in less than 10 seconds.
>
> I have made changes to my local dbunit sources and would be happy to share
> by changes with you. What is the proper procedure?

Great!  Create a tracker entry for each feature/bug fix and attach the
patch file.  The patch will contain the diffs of your prod code
changes and tests that prove the change works.  If you don't know how
to create a patch, let me know your OS, IDE, etc.

Email the dev list after a week or two if there has been no action on
it (i.e. commit) as a reminder.


Thank you for your investigation and forthcoming patch(es)!