xmlunit-general — Everything XMLUnit ;0)

Re: [Xmlunit-general] Problems using schema with Validator

[Stefan B. <bo...@ap...>]

On 2011-03-16, James Abley wrote:

> A brief inspection of the current svn HEAD hasn't shown that there is any
> hard-coded reference to xsd:schema in XMLUnit's source code.

Be sure to look into
<https://xmlunit.svn.sourceforge.net/svnroot/xmlunit/branches/xmlunit-1.x/>
when searching for the source code.  trunk is where I started XMLUnit
2.x development.

Anyway, there shouldn't be any hard-coded references to xsd in XMLUnit
at all.

<https://xmlunit.svn.sourceforge.net/svnroot/xmlunit/branches/xmlunit-1.x/src/java/org/custommonkey/xmlunit/jaxp13/Validator.java>
doesn't show any.

Stefan

Re: [Xmlunit-general] Problems using schema with Validator

[Andy S. <ins...@go...>]

Hi James,

On Wed, 2011-03-16 at 12:37 +0000, James Abley wrote:
> On 16 March 2011 01:42, Andy Stevens <ins...@go...>
> wrote:
>         "Failed to read schema document 'null', because 1) could not
>         find the
>         document; 2) the document could not be read; 3) the root
>         element of the
>         document is not <xsd:schema>".
>         
>         The 'null' I assume is just because I haven't set a specific
>         systemId on the
>         StreamSource.  However, the xsd file is certainly being found
>         (I've checked
>         the resource URL and that's fine) and can certainly be read
>         (I've dumped the
>         InputStream out to the console instead, and see the file's
>         contents).
>         However, the schema file is using "xs" as its prefix rather
>         than "xsd"
>         (with corresponding
>         xmlns:xs="http://www.w3.org/2001/XMLSchema") so the root
>         element is actually <xs:schema> not <xsd:schema>.  So it looks
>         like the
>         error is due to the 3rd option it gives.
...
>         How rigid is the Validator over the prefix being used, given
>         the namespace
>         URL itself is correct?  Is the specific prefix hard-coded
>         somewhere, or is
>         there something I can do to make it check the actual namespace
>         instead?  Or,
>         since the jaxp13.Validator in v1.3 says it can handle a
>         variety of schema
>         languages, is this error not being produced by XMLUnit anyway
>         but by
>         whatever default SchemaFactory it's using?  And if so, is
>         there anything I
>         can do about it? (e.g. a configuration setting I can pass
>         through to the
>         SchemaFactory somehow)  Given I didn't produce the schema
>         files, and the
>         fact there's quite a number of them involved, I'd prefer not
>         to have to go
>         through and find/replace the prefix in them all...
...
> 
> Can you provide a stack trace?
> 
> 
> A brief inspection of the current svn HEAD hasn't shown that there is
> any hard-coded reference to xsd:schema in XMLUnit's source code. 

Sure.  I'd moved on from validating just the schema to try validating a
document instead, but was still seeing the same message in a nested
exception further down the stack trace...

org.custommonkey.xmlunit.exceptions.XMLUnitRuntimeException: Schema is 
invalid
        at 
org.custommonkey.xmlunit.jaxp13.Validator.getInstanceErrors(Validator.java:176)
        at 
fixml.MessageBuilderTest.quickMessageTest(MessageBuilderTest.java:115)
...
Caused by: org.xml.sax.SAXParseException: schema_reference.4: Failed to 
read schema document 'null', because 1) could not find the document; 2) 
the document could not be read; 3) the root element of the document is
not 
<xsd:schema>.
        at 
org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown 
Source)
        at org.apache.xerces.util.ErrorHandlerWrapper.error(Unknown 
Source)
        at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown 
Source)
        at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown 
Source)
        at 
org.apache.xerces.impl.xs.traversers.XSDHandler.reportSchemaError(Unknown 
Source)
        at 
org.apache.xerces.impl.xs.traversers.XSDHandler.getSchemaDocument(Unknown 
Source)
        at 
org.apache.xerces.impl.xs.traversers.XSDHandler.parseSchema(Unknown 
Source)
        at org.apache.xerces.impl.xs.XMLSchemaLoader.loadSchema(Unknown 
Source)
        at
org.apache.xerces.impl.xs.XMLSchemaLoader.loadGrammar(Unknown 
Source)
        at
org.apache.xerces.impl.xs.XMLSchemaLoader.loadGrammar(Unknown 
Source)
        at 
org.apache.xerces.jaxp.validation.XMLSchemaFactory.newSchema(Unknown 
Source)
        at 
org.custommonkey.xmlunit.jaxp13.Validator.parseSchema(Validator.java:202)
        at 
org.custommonkey.xmlunit.jaxp13.Validator.getInstanceErrors(Validator.java:174)

Based on that, I reckon the message is originally coming from Xerces.
But is there anything (e.g. feature or property) I can pass through to
it via the Validator API that might get it to behave better?


Andy

Re: [Xmlunit-general] Problems using schema with Validator

[James A. <jam...@gm...>]

On 16 March 2011 01:42, Andy Stevens <ins...@go...>wrote:

> Hi,
>
> I'm having some trouble using org.custommonkey.xmlunit.jaxp13.Validator,
> and
> I'm wondering if it's a limitation of the validator.
>
> I'm trying to validate some FIXML messages I've generated, using the schema
> files from fixprotocol.org
> (http://fixprotocol.org/specifications/fix4.4fixml).  The schema is split
> over various files using includes, but I don't think that's the problem, as
> you'll see below.  I locate the main file (that ultimately includes the
> rest) using getClass.getResource(), getting the URL's InputStream and
> wrapping that in a StreamSource, and adding that to the Validator as the
> schema source.  However, on attempting to validate the schema (I've not got
> as far as validating the actual XML yet) I get the error message
> "Failed to read schema document 'null', because 1) could not find the
> document; 2) the document could not be read; 3) the root element of the
> document is not <xsd:schema>".
>
> The 'null' I assume is just because I haven't set a specific systemId on
> the
> StreamSource.  However, the xsd file is certainly being found (I've checked
> the resource URL and that's fine) and can certainly be read (I've dumped
> the
> InputStream out to the console instead, and see the file's contents).
> However, the schema file is using "xs" as its prefix rather than "xsd"
> (with corresponding xmlns:xs="http://www.w3.org/2001/XMLSchema") so the
> root
> element is actually <xs:schema> not <xsd:schema>.  So it looks like the
> error is due to the 3rd option it gives.
>
> I tried a search & replace within the file to change the prefix being used,
> but then it just starts complaining about one of the other files (since
> they
> all use the same prefix).  At least that shows that the xsd:include
> elements
> are working okay...
>
> How rigid is the Validator over the prefix being used, given the namespace
> URL itself is correct?  Is the specific prefix hard-coded somewhere, or is
> there something I can do to make it check the actual namespace instead?
>  Or,
> since the jaxp13.Validator in v1.3 says it can handle a variety of schema
> languages, is this error not being produced by XMLUnit anyway but by
> whatever default SchemaFactory it's using?  And if so, is there anything I
> can do about it? (e.g. a configuration setting I can pass through to the
> SchemaFactory somehow)  Given I didn't produce the schema files, and the
> fact there's quite a number of them involved, I'd prefer not to have to go
> through and find/replace the prefix in them all...
>
> Oh yes, I forgot to mention, I'm using IBM's 1.6 JDK (in case that has a
> bearing on the SchemaFactory that will be present) although I suppose I
> could potentially run under any other Windows JDK if that would solve the
> problem more easily.
>
> Can anyone help, or at least steer me in the right direction?
>
>
> Andy
>

Can you provide a stack trace?

A brief inspection of the current svn HEAD hasn't shown that there is any
hard-coded reference to xsd:schema in XMLUnit's source code.

Cheers,

James

[Xmlunit-general] Problems using schema with Validator

[Andy S. <ins...@go...>]

Hi,

I'm having some trouble using org.custommonkey.xmlunit.jaxp13.Validator, and
I'm wondering if it's a limitation of the validator.

I'm trying to validate some FIXML messages I've generated, using the schema
files from fixprotocol.org
(http://fixprotocol.org/specifications/fix4.4fixml).  The schema is split
over various files using includes, but I don't think that's the problem, as
you'll see below.  I locate the main file (that ultimately includes the
rest) using getClass.getResource(), getting the URL's InputStream and
wrapping that in a StreamSource, and adding that to the Validator as the
schema source.  However, on attempting to validate the schema (I've not got
as far as validating the actual XML yet) I get the error message
"Failed to read schema document 'null', because 1) could not find the
document; 2) the document could not be read; 3) the root element of the
document is not <xsd:schema>".

The 'null' I assume is just because I haven't set a specific systemId on the
StreamSource.  However, the xsd file is certainly being found (I've checked
the resource URL and that's fine) and can certainly be read (I've dumped the
InputStream out to the console instead, and see the file's contents).
However, the schema file is using "xs" as its prefix rather than "xsd"
(with corresponding xmlns:xs="http://www.w3.org/2001/XMLSchema") so the root
element is actually <xs:schema> not <xsd:schema>.  So it looks like the
error is due to the 3rd option it gives.

I tried a search & replace within the file to change the prefix being used,
but then it just starts complaining about one of the other files (since they
all use the same prefix).  At least that shows that the xsd:include elements
are working okay...

How rigid is the Validator over the prefix being used, given the namespace
URL itself is correct?  Is the specific prefix hard-coded somewhere, or is
there something I can do to make it check the actual namespace instead?  Or,
since the jaxp13.Validator in v1.3 says it can handle a variety of schema
languages, is this error not being produced by XMLUnit anyway but by
whatever default SchemaFactory it's using?  And if so, is there anything I
can do about it? (e.g. a configuration setting I can pass through to the
SchemaFactory somehow)  Given I didn't produce the schema files, and the
fact there's quite a number of them involved, I'd prefer not to have to go
through and find/replace the prefix in them all...

Oh yes, I forgot to mention, I'm using IBM's 1.6 JDK (in case that has a
bearing on the SchemaFactory that will be present) although I suppose I
could potentially run under any other Windows JDK if that would solve the
problem more easily.

Can anyone help, or at least steer me in the right direction?


Andy

Re: [Xmlunit-general] How can I use an EntityResolver with a Transform?

[miguel <mi...@al...>]

Hi Stefan,

On Sun, 2011-02-27 at 08:12 +0100, Stefan Bodewig wrote:

> 
> At first I thought you'd have to use the URIResolver but this one is
> explicitly only about includes and document() so I don't think it will
> affect the resolution of DOCTYPEs.
> 
> The only way you could do it with JAXP and not XMLUnit is likely by
> setting the EntityResolver on a DocumentBuilder, load the stylesheet
> using this and pass it into the transformation as a DOMSource.
> 
> In fact this should work with XMLUnit as well (using the control builder
> that you have already configured to use your EntityResolver):
> 
>     Document myControlDocument = XMLUnit.buildControlDocument(myControlXML);
>     Document myStyleSheet = XMLUnit.buildControlDocument(new File("src/test/resources/nodeRemover.xsl"));
>     Transform myControlTransform = new Transform(new DOMSource(myControlDocument),
>                                                  new DOMSource(myStyleSheet));
> 

This is exactly the way to go. 
Loading the stylesheet this way will use our own EntityResolver. Thanks
a lot for the information, which saved me hours of desperate search for
the answer!


Miguel

Re: [Xmlunit-general] How can I use an EntityResolver with a Transform?

[Stefan B. <bo...@ap...>]

On 2011-02-25, Miguel wrote:

> - I am using XMLUnit on a set of legacy files with different SYSTEM
> definitions in DOCTYPE. Some of them are invalid (the web address
> changed in real life). Because I had ceated my own EntityResolver to
> resolve to a local file, I decided to use it.

> Now, the problem comes when I try to apply a transformation to one of
> the documents:
> Transform myControlTransform = new Transform(myControlXML, new
> File("src/test/resources/nodeRemover.xsl"));

> This code doesn't seem to use my EntityResolver, but tries to resolve
> the internet address (it doesn't work).

At first I thought you'd have to use the URIResolver but this one is
explicitly only about includes and document() so I don't think it will
affect the resolution of DOCTYPEs.

The only way you could do it with JAXP and not XMLUnit is likely by
setting the EntityResolver on a DocumentBuilder, load the stylesheet
using this and pass it into the transformation as a DOMSource.

In fact this should work with XMLUnit as well (using the control builder
that you have already configured to use your EntityResolver):

    Document myControlDocument = XMLUnit.buildControlDocument(myControlXML);
    Document myStyleSheet = XMLUnit.buildControlDocument(new File("src/test/resources/nodeRemover.xsl"));
    Transform myControlTransform = new Transform(new DOMSource(myControlDocument),
                                                 new DOMSource(myStyleSheet));

Stefan

[Xmlunit-general] How can I use an EntityResolver with a Transform?

[Miguel <mi...@al...>]

Hi,

Consider the following scenario:
- I am using XMLUnit on a set of legacy files with different SYSTEM
definitions in DOCTYPE. Some of them are invalid (the web address
changed in real life). Because I had ceated my own EntityResolver to
resolve to a local file, I decided to use it.

The following lines work:

		XMLUnit.setControlEntityResolver(entityResolver ); //My own
implementation of EntityResolver
		XMLUnit.setTestEntityResolver(entityResolver );
		Document myControlDocument =
XMLUnit.buildControlDocument(myControlXML); //OK, calls my
EntityResolver in the process. all is well
		Document myTestDocument = XMLUnit.buildTestDocument(myTestXML);
		Document myControlDocument =myControlTransform.getResultDocument();
		Document myTestDocument =myTestTransform.getResultDocument();
		XMLUnit.compareXML(myControlDocument,myTestDocument);
		

Now, the problem comes when I try to apply a transformation to one of
the documents:
Transform myControlTransform = new Transform(myControlXML, new
File("src/test/resources/nodeRemover.xsl"));

This code doesn't seem to use my EntityResolver, but tries to resolve
the internet address (it doesn't work). So when I call
myControlTransform.getResultDocument(); I get an empty document instead
of the correct one.

My questions are: 
a) how can I use my EntityResolver with Transform and Transformers?
b) Is it possible to use our own EntityResolver with
XMLUnit.getTransformerFactory()? How?


Thank you very much for your replies,


Miguel Almeida

Re: [Xmlunit-general] Sourcecode

[Stefan B. <bo...@ap...>]

On 2011-02-24, Johannes.Lichtenberger wrote:

> I'm currently searching an XML diff implementation which creates minimal
> edit scripts, but I'm not familiar with the sources of XMLUnit. I think
> XMLUnit must have implemented such an algorithm!?

Not really (it doesn't really strive for minimal).

> If so can someone point me to the main algorithm used -- and the
> concrete class(es)?

The diff algorith used in the released versions is in
<http://xmlunit.svn.sourceforge.net/viewvc/xmlunit/branches/xmlunit-1.x/src/java/org/custommonkey/xmlunit/DifferenceEngine.java?view=log>

A complete re-implementation is in trunk that will end up being XMLUnit
2.0 it I ever get my act together:
Java
<http://xmlunit.svn.sourceforge.net/viewvc/xmlunit/trunk/xmlunit/src/main/java-core/net/sf/xmlunit/diff/DOMDifferenceEngine.java?view=log>
and C#
<http://xmlunit.svn.sourceforge.net/viewvc/xmlunit/trunk/xmlunit/src/main/net-core/diff/DOMDifferenceEngine.cs?view=log>

All of them are purely DOM based and some aspect of it (the
ElementQualifier/NodeMatcher parts) would be pretty complex to do with
SAX or a pull parser setup.  The alogithms themselves don't really use
DOM but could work for any tree structure.

Stefan

[Xmlunit-general] Sourcecode

[Johannes.Lichtenberger <Joh...@un...>]

Hello,

I'm currently searching an XML diff implementation which creates minimal
edit scripts, but I'm not familiar with the sources of XMLUnit. I think
XMLUnit must have implemented such an algorithm!? If so can someone
point me to the main algorithm used -- and the concrete class(es)?

Thanks,
Johannes

Re: [Xmlunit-general] Doubt with ElementQualifier's

[Stefan B. <bo...@ap...>]

On 2011-02-16, Pedro Lemos wrote:

> Hi people,

> I just started using XMLUnit in some of my projects and at the moment I'm
> stuck with the following problem:

> Imagine that I have a control file (control.xml) and a test file (test.xml)
> which I want them to be analyzed as similar.

> When I run the following code:

>         final String control =
>> ResourceUtils.loadResourceString("control.xml");
>>         final String test = ResourceUtils.loadResourceString("test.xml");

>>         final Diff diff = new Diff(control, test);
>>         diff.overrideElementQualifier(new
>> MultiLevelElementNameAndTextQualifier(5, true));
>>         XMLAssert.assertXMLEqual(diff, true);


> The result I get is that the files are similar.

> Although, if I *uncomment *the existent elements, the result is that the
> files are not equal.

> *What do I need to do in my code so that those files be analyzed as being
> similar to each other?*

Figuring out the ElementQualifier is the biggest pain point in current
XMLUnit.  MultiLevelElementNameAndTextQualifier works under very
specific circumstances that are no longer true when you uncomment the
elements - since now suddenly there are more elements with nested text.

There is no built-in ElementQualifier that would be able to select the
Assertion elements you want to compare.  You could write your own that
basically acted as ElementNameQualifier and only if the element name was 
org.openehr.am.archetype.assertion.Assertion then dived down the tree
and compared the pattern elements' text content.

Stefan

Re: [Xmlunit-general] Xalan-J xsl:include and failed URI resolution

[Stefan B. <bo...@ap...>]

On 2011-02-09, Stefan Bodewig wrote:

> I'll consult the Xalan archives and issue tracker.

A quick update:  I spent several hours searching both last night but
couldn't find anything related.  This leads me to suspect there must be
something specific in XMLUnit's testcase and I'll try to distill a test
that is independent of XMLUnit before I make any more noise.

This make some time so people subscribed to the commits list (me) will
have to live with Gump's nagging for now.

Stefan

Re: [Xmlunit-general] Xalan-J xsl:include and failed URI resolution

[Stefan B. <bo...@ap...>]

Hi James,

sorry for the delay but it took me longer than expected until I was able
to recreate an environment similar to Gump on my current development
machine.

On 2011-02-07, James Abley wrote:

>    - Xalan-J has behaved like this for a long time [1].

OK.  This is strange since the trasnformer that is part of th JDK
behaves differently (Sun 1.4 through OpenJDK 7) and AFAIK these should
be forked versions of Xalan.

> What does Saxon do, and perhaps it's worth asking on the xsl-list?

Saxon (9.3HE) causes lots of test failures (unsupported HTML version 1.0
and more things that may be worth investigating at some point) but this
particular test passes.  The ErrorListener receives a fatalError and the
transformer throws the expected exception.

I'll consult the Xalan archives and issue tracker.  If the Sun fork
modifies Xalan at this point and the change never made it back to Xalan
I suspect the Xalan dev folks think their way is correct.

Thanks

        Stefan

Re: [Xmlunit-general] Xalan-J xsl:include and failed URI resolution

[James A. <jam...@gm...>]

Hi Stefan,

I can't really help; just some observations:

   - Xalan-J has behaved like this for a long time [1].
   - Error handling in the spec isn't really defined as well as we would
   like [2]. The HTML5 spec is much nicer in this regard!

What does Saxon do, and perhaps it's worth asking on the xsl-list?

Cheers,

James

[1] https://github.com/apache/xalan-j/commit/308da1a9#L31R150
[2] http://www.w3.org/TR/xslt#element-include

On 7 February 2011 08:50, Stefan Bodewig <bo...@ap...> wrote:

> Hi,
>
> mabye anybody around here has come across this situation.
>
> Recently the ASF Gump instance has started to make sure XMLUnit's unit
> tests run on Xerces and Xalan rather than the parser/transformer of the
> JDK.  This is when the tests started to fail, the only remaining failure
> can be seen here:
>
>
> http://vmgump.apache.org/gump/public/xmlunit/xmlunit-test/gump_file/TEST-org.custommonkey.xmlunit.test_Transform.xml.html
>
> The test uses a stylesheet containing an xsl:include that cannot be
> resolved and expects the transformer to throw an exception.  In Xalan's
> case this does not happen.  As can be seen in the log the error and not
> the fatalError message of the attached ErrorListener is invoked so it
> seems Xalan doesn't consider the failing include fatal.
>
> The JAXP docs on URI resolver aren't really clear on what is to be
> expected if the resolution fails (or maybe I'm just not finding the
> right doc).  Does anybody know whether it is OK for Xalan to treat the
> failing include as non-fatal or whether I should report a bug?
>
> Stefan
>
>
> ------------------------------------------------------------------------------
> The modern datacenter depends on network connectivity to access resources
> and provide services. The best practices for maximizing a physical server's
> connectivity to a physical network are well understood - see how these
> rules translate into the virtual world?
> http://p.sf.net/sfu/oracle-sfdevnlfb
> _______________________________________________
> Xmlunit-general mailing list
> Xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlunit-general
>

[Xmlunit-general] Xalan-J xsl:include and failed URI resolution

[Stefan B. <bo...@ap...>]

Hi,

mabye anybody around here has come across this situation.

Recently the ASF Gump instance has started to make sure XMLUnit's unit
tests run on Xerces and Xalan rather than the parser/transformer of the
JDK.  This is when the tests started to fail, the only remaining failure
can be seen here:

http://vmgump.apache.org/gump/public/xmlunit/xmlunit-test/gump_file/TEST-org.custommonkey.xmlunit.test_Transform.xml.html

The test uses a stylesheet containing an xsl:include that cannot be
resolved and expects the transformer to throw an exception.  In Xalan's
case this does not happen.  As can be seen in the log the error and not
the fatalError message of the attached ErrorListener is invoked so it
seems Xalan doesn't consider the failing include fatal.

The JAXP docs on URI resolver aren't really clear on what is to be
expected if the resolution fails (or maybe I'm just not finding the
right doc).  Does anybody know whether it is OK for Xalan to treat the
failing include as non-fatal or whether I should report a bug?

Stefan

Re: [Xmlunit-general] XmlUnit throws FileNotFoundException when creating a Diff with 2 log4j.xml files (can't find log4j.dtd file)

[Stefan B. <bo...@ap...>]

On 2011-01-20, Dean Schulze wrote:

> When I create a Diff with two log4j.xml files it throws a
> FileNotFoundException with this message:

> java.io.FileNotFoundException:
> C:\dean\projects\ear.expand.diff\log4j.dtd (The system cannot find the
> file specified)

> Apparently XmlUnit is looking for log4j.dtd in the directory where I'm
> executing XmlUnit.

> I don't want XmlUnit to validate the 2 log4j.xml files.  I only want
> to know if they are similar.

XMLUnit itself doesn't try to validate the documents per se, but your
XML parser might want to do so.

> Do I have to turn off validation in this case?

Yes.

If you are configuring the DocumentBuilder XMLUnit uses yourself, make
sure you don't enable validation.  If you do not configure it yourself
than the default created by your JVM is in validating mode.  In order to
turn validation off you'd have to create an instance of the default
DocumentBuilderFactory, set it to not validate XML and pass it to both
XMLUnit.setControlDocumentBuilderFactory and
setTestDocumentBuildFactory.

Alternatively you can create the DOM Document instances yourself and use
the Diff constructor (or the XMLunit.compareXML overload) taking two
Documents.

Stefan

[Xmlunit-general] XmlUnit throws FileNotFoundException when creating a Diff with 2 log4j.xml files (can't find log4j.dtd file)

[Dean S. <dea...@ya...>]

When I create a Diff with two log4j.xml files it throws a FileNotFoundException with this message:

java.io.FileNotFoundException: C:\dean\projects\ear.expand.diff\log4j.dtd (The system cannot find the file specified)


Apparently XmlUnit is looking for log4j.dtd in the directory where I'm executing XmlUnit.

I don't want XmlUnit to validate the 2 log4j.xml files.  I only want to know if they are similar.

Do I have to turn off validation in this case?

Thanks.





      

Re: [Xmlunit-general] Sequence and text difference confusion

[Jonathan F. <Jon...@or...>]

Thank you very much! Actually I wanted that behaviour for all elements - I was just looking at application-variable as an example. But I looked into the ElementQualifier and it seems like RecursiveElementNameAndTextQualifier does exactly what I want - I was wanting recursive comparisons for all elements. Thank you again for explaining about ElementQualifiers.

Yours Sincerely,
Jonathan Foulkes

----- Forwarded Message -----
From: "Stefan Bodewig" <bo...@ap...>
To: xml...@li...
Sent: Tuesday, December 14, 2010 11:26:20 PM
Subject: Re: [Xmlunit-general] Sequence and text difference confusion

Hi Jonathan

On 2010-12-14, Jonathan Foulkes wrote:

> I'm working on a project that involves producing visual diffs of XML
> files. A major requirement is that elements should be able to be
> re-ordered but will not display as different unless they are logically
> different. I thought that XMLUnit's Diff class would accomplish this
> using the similar() function, and built my application
> accordingly.

In general it does - as long as it compares the elements you want it to
compare.  Sometimes XMLUnit will need you help in order to know what is
important to your comparision and this help comes by an
<http://xmlunit.sourceforge.net/userguide/html/ar01s03.html#ElementQualifier>

> For example, I have two blocks of XML:
> <application-variable>
>   <name>database</name>
>   <protected>false</protected>
>   <value>SupportTracker</value>
> </application-variable>
> <application-variable>
>   <name>conserver</name>
>   <protected>false</protected>
>   <value>jonathanf-vm</value>
> </application-variable>

> and

> <application-variable>
>   <name>conserver</name>
>   <protected>false</protected>
>   <value>jonathanf-vm</value>
> </application-variable>
> <application-variable>
>   <name>database</name>
>   <protected>false</protected>
>   <value>SupportTracker</value>
> </application-variable>

By default XMLUnit will compare the first application-variable element
of either XML snippet with each other and they are identical (same name,
no attributes at all, same number of children) and the dive further down
the subtree rooted by this element.  Of course the name elements it sees
next are different.

What you want XMLUnit to do is to make it realize that it has to look at
the nested text of the "name" child element of the
"application-variable" element when it decides which subtrees to
compare.

Neither of the built-in ElementQualifiers will work for you, you will
have to write an implementation of your own - but that won't be too
difficult.  You could extend ElementNameQualifier and only intercept it
if the element's local name matches application-variable to inject you
own logic.

Stefan

------------------------------------------------------------------------------
Lotusphere 2011
Register now for Lotusphere 2011 and learn how
to connect the dots, take your collaborative environment
to the next level, and enter the era of Social Business.
http://p.sf.net/sfu/lotusphere-d2d
_______________________________________________
Xmlunit-general mailing list
Xml...@li...
https://lists.sourceforge.net/lists/listinfo/xmlunit-general

Re: [Xmlunit-general] Sequence and text difference confusion

[Stefan B. <bo...@ap...>]

Hi Jonathan

On 2010-12-14, Jonathan Foulkes wrote:

> I'm working on a project that involves producing visual diffs of XML
> files. A major requirement is that elements should be able to be
> re-ordered but will not display as different unless they are logically
> different. I thought that XMLUnit's Diff class would accomplish this
> using the similar() function, and built my application
> accordingly.

In general it does - as long as it compares the elements you want it to
compare.  Sometimes XMLUnit will need you help in order to know what is
important to your comparision and this help comes by an
<http://xmlunit.sourceforge.net/userguide/html/ar01s03.html#ElementQualifier>

> For example, I have two blocks of XML:
> <application-variable>
>   <name>database</name>
>   <protected>false</protected>
>   <value>SupportTracker</value>
> </application-variable>
> <application-variable>
>   <name>conserver</name>
>   <protected>false</protected>
>   <value>jonathanf-vm</value>
> </application-variable>

> and

> <application-variable>
>   <name>conserver</name>
>   <protected>false</protected>
>   <value>jonathanf-vm</value>
> </application-variable>
> <application-variable>
>   <name>database</name>
>   <protected>false</protected>
>   <value>SupportTracker</value>
> </application-variable>

By default XMLUnit will compare the first application-variable element
of either XML snippet with each other and they are identical (same name,
no attributes at all, same number of children) and the dive further down
the subtree rooted by this element.  Of course the name elements it sees
next are different.

What you want XMLUnit to do is to make it realize that it has to look at
the nested text of the "name" child element of the
"application-variable" element when it decides which subtrees to
compare.

Neither of the built-in ElementQualifiers will work for you, you will
have to write an implementation of your own - but that won't be too
difficult.  You could extend ElementNameQualifier and only intercept it
if the element's local name matches application-variable to inject you
own logic.

Stefan

[Xmlunit-general] Sequence and text difference confusion

[Jonathan F. <Jon...@or...>]

Hi All,

I'm working on a project that involves producing visual diffs of XML files. A major requirement is that elements should be able to be re-ordered but will not display as different unless they are logically different. I thought that XMLUnit's Diff class would accomplish this using the similar() function, and built my application accordingly. However, when I tested this, I found that reordered elements were instead being identified as non-recoverable TEXT_VALUE differences.

For example, I have two blocks of XML:
<application-variable>
  <name>database</name>
  <protected>false</protected>
  <value>SupportTracker</value>
</application-variable>
<application-variable>
  <name>conserver</name>
  <protected>false</protected>
  <value>jonathanf-vm</value>
</application-variable>

and 

<application-variable>
  <name>conserver</name>
  <protected>false</protected>
  <value>jonathanf-vm</value>
</application-variable>
<application-variable>
  <name>database</name>
  <protected>false</protected>
  <value>SupportTracker</value>
</application-variable>

These are logically equivalent (similar). However, when I compare these, I get:
Expected text value 'conserver' but was 'database' - comparing conserver at /concerto[1]/application[1]/application-variable[1]/name[1]/text()[1] to database at /concerto[1]/application[1]/application-variable[1]/name[1]/text()[1] 
    
Expected text value 'jonathanf-vm' but was 'SupportTracker' - comparing jonathanf-vm at /concerto[1]/application[1]/application-variable[1]/value[1]/text()[1] to SupportTracker at /concerto[1]/application[1]/application-variable[1]/value[1]/text()[1] 

Expected text value 'database' but was 'conserver' - comparing database at /concerto[1]/application[1]/application-variable[2]/name[1]/text()[1] to conserver at /concerto[1]/application[1]/application-variable[2]/name[1]/text()[1] 

Expected text value 'SupportTracker' but was 'jonathanf-vm' - comparing SupportTracker at /concerto[1]/application[1]/application-variable[2]/value[1]/text()[1] to jonathanf-vm at /concerto[1]/application[1]/application-variable[2]/value[1]/text()[1]

Is there a way of comparing XML files such that these logical similarities are preserved? Am I just not setting a configuration option? Any advice would be much appreciated.

Yours Sincerely,
Jonathan Foulkes 
 

Re: [Xmlunit-general] DetailedDiff appears to compare staggered notes instead of exact xpath location

[Stefan B. <bo...@ap...>]

On 2010-10-05, David Pesce wrote:

> I'm not sure exactly why, but DetailedDiff appears to be comparing
> different nodes between files.

For many use cases node order is not so significant and so XMLUnit
compares elements out of order if necessary.  For Diff you can solve
that by overriding the DifferenceListener with one that upgrades
CHILD_NODELIST_SEQUENCE differences to non-recoverable.  This will make
Diff stop once it detects elements out of order - it won't help with
DetailedDiff.

Make sure you set the compareUnmatched parameter to false.

> I've even tried setting the ElementQualifier to null in both the Diff
> and DetailedDiff:

The problem here is that XMLUnit 1.x will always attempt to match nodes
regardless of order.  If you set ElementQualifier to null it should
visit elements in correct order, unless there are other node types (like
emoty text nodes) between them that throws its comparison algorithm out
of step.

Can you provide two example documents and say what you'd expect XMLUnit
to return so we have something more concrete to talk about?

Stefan

[Xmlunit-general] DetailedDiff appears to compare staggered notes instead of exact xpath location

[David P. <dav...@gm...>]

JRE 1.6
XMLUnit 1.3
Windows 7

I'm not sure exactly why, but DetailedDiff appears to be comparing different
nodes between files. I'd like it to compare the exact xpath location in one
file to the exact xpath location in the other.

Output example to explain a bit:

This one looks ok:
Expected attribute name 'product' but was 'null' - comparing <topic...> at
/topic[1] to <topic...> at /topic[1]

Here it is comparing (at the end of the xpath loc) node 2 to node 3:
Expected attribute name 'null' but was 'product' - comparing <li...> at
/topic[1]/body[1]/section[1]/table[1]/tbody[1]/tr[2]/td[2]/ul[1]/li[2] to
<li...> at
/topic[1]/body[1]/section[1]/table[1]/tbody[1]/tr[2]/td[2]/ul[1]/li[3]

Same thing here (node 3 to node 4)
Expected attribute value 'test' but was 'test2' - comparing <li
product="test"...> at
/topic[1]/body[1]/section[1]/table[1]/tbody[1]/tr[2]/td[2]/ul[1]/li[3]/@product
to <li product="test2"...> at
/topic[1]/body[1]/section[1]/table[1]/tbody[1]/tr[2]/td[2]/ul[1]/li[4]/@product

I've even tried setting the ElementQualifier to null in both the Diff and
DetailedDiff:
Diff xmlDiff = new Diff(xmlStream, xmlStream2);
 xmlDiff.overrideElementQualifier(null);
DetailedDiff dXmlDiff = new DetailedDiff(xmlDiff);
 dXmlDiff.overrideElementQualifier(null);

Any help would be greatly appreciated.
Dave

Re: [Xmlunit-general] Can one ignore empty tags when comparing

[Stefan B. <bo...@ap...>]

On 2010-09-28, Miguel wrote:

> I have a doubt though: I'm trying to compare 2 XML files for their
> content. This means that:

> 		<primarysource>
> 			<reportertitle>Dr</reportertitle>
> 			<reportergivename>Antonio</reportergivename>
> 			<reportermiddlename></reportermiddlename>
> 			<reporterfamilyname>Perez</reporterfamilyname>
>                   <primarysource>

> and
>                 <primarysource>
>                         <reportertitle>Dr</reportertitle>
>                         <reportergivename>Antonio</reportergivename>
>                         <reporterfamilyname>Perez</reporterfamilyname>
>                   <primarysource>

> Are similar, because the non-null/empty content is essentially the same.

> Is there an option on xmlunit to make the Diff engine ignore empty tags
> so that these XML are calculated to be similar?

No, no built-in option.  You could write an implementation of
DifferenceListener
<http://xmlunit.sourceforge.net/userguide/html/ar01s03.html#DifferenceListener>
which would check whether the Node that is part of a CHILD_NODE_NOT
difference has an empty text content and return
RETURN_IGNORE_DIFFERENCE_NODES_SIMILAR in that case.

You will also want to disable compareUnmateched
<http://xmlunit.sourceforge.net/userguide/html/ar01s03.html#Comparison%20of%20Unmatched%20Elements>

Stefan

[Xmlunit-general] Can one ignore empty tags when comparing

[Miguel <mi...@al...>]

First of all, congratulations on the project, which I am starting to use
and seems to be very interesting.

I have a doubt though: I'm trying to compare 2 XML files for their
content. This means that:

		<primarysource>
			<reportertitle>Dr</reportertitle>
			<reportergivename>Antonio</reportergivename>
			<reportermiddlename></reportermiddlename>
			<reporterfamilyname>Perez</reporterfamilyname>
                  <primarysource>

and
                <primarysource>
                        <reportertitle>Dr</reportertitle>
                        <reportergivename>Antonio</reportergivename>
                        <reporterfamilyname>Perez</reporterfamilyname>
                  <primarysource>

Are similar, because the non-null/empty content is essentially the same.

Is there an option on xmlunit to make the Diff engine ignore empty tags
so that these XML are calculated to be similar?

Thank you for your answer,

Miguel Almeida

[Xmlunit-general] Revisiting Node Selection

[Stefan B. <bo...@ap...>]

Hi,

most if not all bug reports and user questions somehow boil down to
finding the best strategy to make XMLUnit pick matching elements - and
the 1.3 code isn't always obvious.

I've changed trunk so that the whole strategy of selecting the best
matching nodes among the children of test and control elements is
pluggable:

    /**
     * Matches control and test nodes against each other, returns the
     * matching pairs.
     */
    Iterable<Map.Entry<Node, Node>> match(Iterable<Node> controlNodes,
                                          Iterable<Node> testNodes);

and

    IEnumerable<KeyValuePair<XmlNode, XmlNode>>
        Match(IEnumerable<XmlNode> controlNodes,
              IEnumerable<XmlNode> testNodes);


I'm thinking of making the parent Node or its XPath (or both) available
as well.

The default strategy picks non-element nodes in order if they are of the
same type (where CDATA and Text are the same type) and uses an
ElementSelector (2.x term for ElementQualifier) for elements.

The legacy layer can replace this with a strategy that adds the
compareUnmatched logic.

I'm also thinking of making ElementSelector allow some sort of middle
ground where it can say "yes the nodes match" and "I know the nodes
don't match" as well as "they don't really look like my business".  For
example if an ElementNameAndAttributeQualifier is asked to compare
elements that don't have any attributes.  This may allow us to pick a
fallback strategy.

It may even be possible to provide a builder with a conditional
strategy, something like "for foo elements, check the bar attribute -
for baz elements compare the nested text of xyzzy elements" which might
look like

    NodeMatcher nm = NodeMatcherBuilder.setDefault(ElementSelectors.byName)
        .if(ElementName.is("foo"))
        .use(ElementSelectors.byNameAndAttributes("bar"))
        .if(ElementName.is("baz"))
        .use(ElementSelectors.byNameAndNestedTextRec(".//xyzzy"))
        .build();

Does that sound reasonable?

Stefan

[Xmlunit-general] Complex ElementQualifiers Work by Accident Only

[Stefan B. <bo...@ap...>]

Hi,

XMLUnit's trunk is almost complete from a functional point of view to
start thinking about XMLUnit 2.0 (with .NET and Java getting released in
parallel), but that's news for a different thread.

While trying to reimplement the Java 1.3 API on top of XMLUnit 2.x's
code I learned that the reason why ElementQualifier works is that
XMLUnit 1.3 sometimes ignores its results - which is counterintuitive.

The best example is RecursiveElementAndTextQualifier.  It is supposed to
help with something like this

control:
<a>
  <b>
    <c>foo</c>
  </b>
  <b>
    <c>bar</c>
  </b>
</a>

test:
<a>
  <b>
    <c>bar</c>
  </b>
  <b>
    <c>foo</c>
  </b>
</a>

where the user expects the documents to be similar and the only
difference would be that the two b's are swapped.

RecursiveElementAndTextQualifier makes two elements eligible for
comparison if the element name matches and all its decendent element
names match (and there is no more than a single element child per level)
and the nested text of the leaf element descendent match.  This works
well for the <b>s.

The problem here is, with the trunk code you won't even get there.
RecursiveElementAndTextQualifier will say the <a>s don't match.

Unless you set compareUnmatched to false in XMLUnit 1.3 it still works
because XMLUnit 1.3 will ignore the result and simply compare the two
<a>s since they are the only nodes available anyway.

This means compareUnmatched=false can make your ElementQualifier fail in
XMLUnit 1.3.

It also means I have to rethink how ElementSelectors should work in
XMLUnit 2.x since I feel compareUnmatched=true is a bad idea and don't
intend to support it in XMLUnit 2.x at all.  Any ideas?

Stefan