Re: [dbunit-developer] Extended flat xml format

[Jeff J. <jj...@ap...>]

On Sun, Feb 13, 2011 at 2:07 PM, Roberto Lo Giacco
<rlo...@us...> wrote:
>
>
> On Sun, Feb 13, 2011 at 18:30, Jeff Jensen <jj...@ap...> wrote:
>>
>> On Sat, Feb 12, 2011 at 4:55 PM, Roberto Lo Giacco <rlo...@gm...>
>> wrote:
>> > What do you think about this extended
>> > notation? http://pastebin.com/ZNifaJY4
>>
>> So auth is schema, user is table as 2.x?
>> I don't understand purpose of <index />
>
> Yes, auth is schema and user is table as per 2.x flat xml format.
> The index or ext:index is a way to avoid having to define row by row the
> sequence number and providing an easy way to retrieve the correct sequence
> number using a more remindable search field like username or group name.
> Actually you have to compare that file against the one committed in the
> subversion repository which has an id column on both user and group.
> An additional ext:property can provide an easy way to define some constant
> value without having to repeatedly copy it to every row.
>
> Using this flat xml format extensions we can add missing features without
> breaking old datasets compatibility.
>>
>> > The counterside is that no `index` table can be defined, unless we use a
>> > custom namespace, which will be more verbose but
>> > cleaner http://pastebin.com/whfMPJYs
>>
>> I don't understand purpose of <ext:index/>, as above.
>
> I hope it is clear now.
>
>>
>> Otherwise, it seems same as 2.x but has some type of embedded
>> variables the dbUnit will process(?).
>
> May be it is worth to you to give a look at our last meeting notes. We are
> suggesting to introduce some sort of EL to allow dbunit to have some
> intelligence in the datasets. As a consequence we will be able to use ${now}
> or ${env.USER_HOME} or other similar constructs... which seems something
> many of our users asked for. As identity columns are not a very spread
> feature among rdbms and we wish to have database agnostic datasets we where
> suggesting to introduce something like ${sequence} which is capable to
> produce identity columns.
>
>>
>> > BTW, trying to use JSON I didn't find any improvement against flat-xml
>> > datasets, do we really need that format?
>>
>> I don't yet see a need for JSON format.  However, I'm fine if that was
>> the default.
>
> We thought that supporting JSON could improve datasets compactness, but as I
> said, it doesn't seem to help in any way
>
>>
>> > I was thinking about having one file format only (flat-xml): the
>> > flat-xml
>> > format with the `ext` namespace can give us the flexibility of the full
>> > blown xml with compactness of JSON.
>>
>> In all my customer work, I've only used the flat XML format, so I am
>> quite biased! :-)
>
> I've always used flat xml format too, but searching our tracker we found
> reports regarding the Excel format, the CSV format and some suggestions to
> use the full blown XML format to avoid null replacements... The latter can
> be solved using something like ${null}.
>
>>
>> > We can eventually provide
>> > converters which can still be handy for Excel editing and CSV import...
>>
>> I think a spreadsheet format is very handy to support, based on no
>> facts at my disposal. :-)  I imagine the spreadsheet format would be
>> most useful when working on tests with users.
>
> Do you think converters toward our XML format will suffice the need or we
> should support multiple formats?

I think we start with one and then add.


>> > The
>> > main difference will be that we will have the possibility to add new
>> > features without worrying about all the file formats: such `ext`
>> > namespace
>> > features will be available in xml types only. Imagine how less issues we
>> > will have in the tracker not having all those "this feature is available
>> > in
>> > XML file format only" or "use full blown XML instead of flat XML dataset
>> > format" and similar...
>>
>> Reducing support needs means reducing opportunity for problems, so I
>> am happy with just flat XML.  I don't care to use the full XML format
>> at all.
>
> Well, many people think the full blown XML format is too verbose... and I
> almost agree with them.

I do agree with them! :-)


>> We can target that for the first releases and add new formats we
>> determined (to me that means either enough requests/votes or patches).
>
> Well, if we decide to go for one main format and converters we can change a
> little bit the design to better fit with this decision and we can add more
> and more features to the XML format. Otherwise we should be careful as we
> need to provide a common set of features for different formats.
> With a single xml format we can avoid having properties and features using
> the xml file to provide any setting we need. The same is harder to achieve
> with other formats like CSV or XSL... Do you see my point?

Yes, I follow you.  I think we need to abstract the feature, even if
we plan for one.  e.g. Strategy pattern designed with interfaces
(stating the obvious!).  Then it becomes much more easily
substitutable by us or contributors...
We can try to make some shared processing logic (e.g. processing of
some literals/variables) for the implementations to share.  But I
don't expect feature-to-feature compatibility per implementation (e.g.
XML and CSV supporting same variables/functions); we can try but I
wouldn't make it the requirement.  This way we can take advantage of
each format's strengths.