xmlunit-general

[Xmlunit-general] Schema Validation patch development

[James A. <jam...@gm...>]

I'm writing a patch for consideration that will handle schema validation.

My use case is as follows:

An XML document is generated in some fashion, but it does not contain

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="somenamespace somelocation"

So I would like to set some properties on a Validator like this:

validator = new Validator(new FileReader(xmlFile));
validator.setSchemaNamespace("http://www.example.com/xmlns/books");
validator.setSchemaURL(xsdFile.toURL());
validator.useXMLSchema(true);

validator.assertIsValid();

This is what my patch is aiming to provide. Comments on the API and
intended usage are welcome.

In developing the tests for this patch though, I seem to have hit a
bug, and was hoping someone could help me see what I'm doing wrong.
The below tests fail, although I don't think they should. I can't see
what properties I should be setting that would cause it to pass.


import org.custommonkey.xmlunit.DetailedDiff;
import org.custommonkey.xmlunit.Diff;
import org.custommonkey.xmlunit.XMLTestCase;

public class XMLUnitNamespaceTestCase extends XMLTestCase {

    private String expected = "<root
xmlns='http://www.example.com/xmlns/example'"
        + " xmlns:xsi='http://exampe.com/xmlns/not/schema-instance/namespace'
/>";;
    private String actual =  "<root
xmlns='http://www.example.com/xmlns/example' />";

    public void testNamespaceDifferencesFailViaAssertXMLMethod()
throws Exception {
        assertXMLNotEqual(expected, actual);
    }

    public void testNamespaceDifferencesFailViaDetailedDiff() throws Exception {
        Diff diff = compareXML(expected, actual);
        DetailedDiff detailedDiff = new DetailedDiff(diff);
        assertFalse(detailedDiff.similar());
    }
}

Cheers,

James

Re: [Xmlunit-general] Schema Validation patch development

[Stefan B. <bo...@ap...>]

On Thu, 21 Dec 2006, James Abley <jam...@gm...> wrote:

> I'm writing a patch for consideration that will handle schema
> validation.

[please use CVS diff to create patches, it is easier to see what has
changed that way]

> So I would like to set some properties on a Validator like this:
> 
> validator = new Validator(new FileReader(xmlFile));
> validator.setSchemaNamespace("http://www.example.com/xmlns/books");
> validator.setSchemaURL(xsdFile.toURL());
> validator.useXMLSchema(true);
> 
> validator.assertIsValid();

This is something you'd use a catalog for. See OASIS[1] and the XML
resolver in xml-commons at Apache.  Ant supports a simple Map based
catalog internally as well as one based on the XML resolver.  I was
thinking about adding this in a similar way to XMLUnit.

> This is what my patch is aiming to provide. Comments on the API and
> intended usage are welcome.

rather than pairs of setSchemaNamespace/setSchemaURI I'd prefer a
single method, something like

public void setSchemaLocation(String nsURI, URL nsURL)

or some kind of Map or - see above - a full catalog implemention.

> In developing the tests for this patch though, I seem to have hit a
> bug, and was hoping someone could help me see what I'm doing wrong.
> The below tests fail, although I don't think they should. I can't
> see what properties I should be setting that would cause it to pass.

>     private String expected = "<root xmlns='http://www.example.com/xmlns/example'" + "
>         + xmlns:xsi='http://exampe.com/xmlns/not/schema-instance/namespace'/>";;
>     private String actual =  "<root xmlns='http://www.example.com/xmlns/example' />";

Why do you expect them to be different?  The
http://exampe.com/xmlns/not/schema-instance/namespace is never used.

Cheers

        Stefan

Footnotes: 
[1]  http://www.oasis-open.org/committees/entity/spec-2001-08-06.html

Re: [Xmlunit-general] Schema Validation patch development

[James A. <jam...@gm...>]

This bounced, so trying again.

---------- Forwarded message ----------
From: James Abley <jam...@gm...>
Date: 24-Dec-2006 09:49
Subject: Re: [Xmlunit-general] Schema Validation patch development
To: xml...@li...


On 22/12/06, Stefan Bodewig <bo...@ap...> wrote:
> On Thu, 21 Dec 2006, James Abley <jam...@gm...> wrote:
>
> > I'm writing a patch for consideration that will handle schema
> > validation.
>
> [please use CVS diff to create patches, it is easier to see what has
> changed that way]

Will do - I'm just TDDing the feature at the moment and wanted to give
a little context.

>
> > So I would like to set some properties on a Validator like this:
> >
> > validator = new Validator(new FileReader(xmlFile));
> > validator.setSchemaNamespace("http://www.example.com/xmlns/books");
> > validator.setSchemaURL(xsdFile.toURL());
> > validator.useXMLSchema(true);
> >
> > validator.assertIsValid();
>
> This is something you'd use a catalog for. See OASIS[1] and the XML
> resolver in xml-commons at Apache.  Ant supports a simple Map based
> catalog internally as well as one based on the XML resolver.  I was
> thinking about adding this in a similar way to XMLUnit.

Agreed. It will probably get to that level of complexity once I've got
an initial implementation in place.

>
> > This is what my patch is aiming to provide. Comments on the API and
> > intended usage are welcome.
>
> rather than pairs of setSchemaNamespace/setSchemaURI I'd prefer a
> single method, something like
>
> public void setSchemaLocation(String nsURI, URL nsURL)
>
> or some kind of Map or - see above - a full catalog implemention.

As above - I'm just getting the basic refactoring and API change in
for now, but the catalog would be my preferred full implementation.

>
> > In developing the tests for this patch though, I seem to have hit a
> > bug, and was hoping someone could help me see what I'm doing wrong.
> > The below tests fail, although I don't think they should. I can't
> > see what properties I should be setting that would cause it to pass.
>
> >     private String expected = "<root xmlns='http://www.example.com/xmlns/example'" + "
> >         + xmlns:xsi='http://exampe.com/xmlns/not/schema-instance/namespace'/>";;
> >     private String actual =  "<root xmlns='http://www.example.com/xmlns/example' />";
>
> Why do you expect them to be different?  The
> http://exampe.com/xmlns/not/schema-instance/namespace is never used.
>

Well, because it has the namespace declaration and to my current way
of thinking, that's just an attribute with special semantics. I think
I'll need to dive into the XML (1.0 only for now) and XML Namespaces
specs to see what they discuss and maybe re-evaluate my defintion of
equality in this case.

As part of my current implementation, it examines the XML that's being
validated for the standard schema instance namespace and a
schemaLocation attribute in that namespace, which the parser can use
as a hint when doing schema validation.

One of my tests was checking that the namespace declaration was added
correctly to the document if it's not already present, but xmlunit
doesn't suppport my assumption of how it would behave. I'm also
looking at the current implementation and it does a DOM comparison,
and maybe a SAX-record-and-replay implementation would allow more
options to be configured for this, similar to the way oXygen allows
XML diff options. That would be a bigger change though!

Cheers,

James


PS - just realised that the mails on this list don't have the Reply-To
header set to the mailing list, hence the reason I've been replying to
individuals rather than the list. Might it be possible to alter that?

> Cheers
>
>        Stefan
>
> Footnotes:
> [1]  http://www.oasis-open.org/committees/entity/spec-2001-08-06.html
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Xmlunit-general mailing list
> Xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlunit-general
>

Re: [Xmlunit-general] Schema Validation patch development

[Stefan B. <bo...@ap...>]

On Tue, 2 Jan 2007, James Abley <jam...@gm...> wrote:
> On 22/12/06, Stefan Bodewig <bo...@ap...> wrote:
>> On Thu, 21 Dec 2006, James Abley <jam...@gm...> wrote:
>>
>> > I'm writing a patch for consideration that will handle schema
>> > validation.
>>
>> [please use CVS diff to create patches, it is easier to see what
>> has changed that way]
> 
> Will do - I'm just TDDing the feature at the moment and wanted to
> give a little context.

That's fine.  Thank you.

>> > In developing the tests for this patch though, I seem to have hit
>> > a bug, and was hoping someone could help me see what I'm doing
>> > wrong.  The below tests fail, although I don't think they
>> > should. I can't see what properties I should be setting that
>> > would cause it to pass.
>>
>> >     private String expected = "<root
>> >     xmlns='http://www.example.com/xmlns/example'" + "
>> >         + xmlns:xsi='http://exampe.com/xmlns/not/schema-instance/namespace'/>";;
>> >     private String actual =  "<root xmlns='http://www.example.com/xmlns/example' />";
>>
>> Why do you expect them to be different?  The
>> http://exampe.com/xmlns/not/schema-instance/namespace is never
>> used.
>>
> 
> Well, because it has the namespace declaration and to my current way
> of thinking, that's just an attribute with special semantics.

The way XMLUnit treats it it is not an attribute at all.  This has a
few benefits like allowing to documents to be equal if they use the
same namespaces but chose different prefixes for example.

If you declare a namespace but never use it, this will never be seen
by XMLUnit since (as you've seen) this doesn't affect the DOM tree at
all.

Do you think this is a problem?

> I just realised that the mails on this list don't have the
> Reply-To header set to the mailing list, hence the reason I've been
> replying to individuals rather than the list. Might it be possible
> to alter that?

I'll note that setting ot not setting Reply-To headers is the subject
of heated discussions (just do a search for "reply-to considered
harmful" with your favorite search engine).  Fortunately I use a MUA
that deals with either way so I'm pretty indifferent.

I don't have karma to change that setting, though.  Jeff?

Cheers

        Stefan

Re: [Xmlunit-general] Schema Validation patch development

[James A. <jam...@gm...>]

On 05/01/07, Stefan Bodewig <bo...@ap...> wrote:
> On Tue, 2 Jan 2007, James Abley <jam...@gm...> wrote:
> > On 22/12/06, Stefan Bodewig <bo...@ap...> wrote:
> >> On Thu, 21 Dec 2006, James Abley <jam...@gm...> wrote:
> >>
> >> > I'm writing a patch for consideration that will handle schema
> >> > validation.
> >>
> >> [please use CVS diff to create patches, it is easier to see what
> >> has changed that way]
> >
> > Will do - I'm just TDDing the feature at the moment and wanted to
> > give a little context.
>
> That's fine.  Thank you.
>
> >> > In developing the tests for this patch though, I seem to have hit
> >> > a bug, and was hoping someone could help me see what I'm doing
> >> > wrong.  The below tests fail, although I don't think they
> >> > should. I can't see what properties I should be setting that
> >> > would cause it to pass.
> >>
> >> >     private String expected = "<root
> >> >     xmlns='http://www.example.com/xmlns/example'" + "
> >> >         + xmlns:xsi='http://exampe.com/xmlns/not/schema-instance/namespace'/>";;
> >> >     private String actual =  "<root xmlns='http://www.example.com/xmlns/example' />";
> >>
> >> Why do you expect them to be different?  The
> >> http://exampe.com/xmlns/not/schema-instance/namespace is never
> >> used.
> >>
> >
> > Well, because it has the namespace declaration and to my current way
> > of thinking, that's just an attribute with special semantics.
>
> The way XMLUnit treats it it is not an attribute at all.  This has a
> few benefits like allowing to documents to be equal if they use the
> same namespaces but chose different prefixes for example.
>
> If you declare a namespace but never use it, this will never be seen
> by XMLUnit since (as you've seen) this doesn't affect the DOM tree at
> all.
>
> Do you think this is a problem?

It is a little annoying when I'm trying to write comprehensive tests.
I've got around it by adding a flag to XMLUnit that prevents any xmlns
attributes from being ignored, but I would probably want to do that in
a more robust fashion as you suggest. If namespaces are being
compared, then  (probably) the namespace name is the important thing
to compare, rather than the prefixes. I'll write some tests around
this area and see how it goes (in a separate patch though).

>
> > I just realised that the mails on this list don't have the
> > Reply-To header set to the mailing list, hence the reason I've been
> > replying to individuals rather than the list. Might it be possible
> > to alter that?
>
> I'll note that setting ot not setting Reply-To headers is the subject
> of heated discussions (just do a search for "reply-to considered
> harmful" with your favorite search engine).  Fortunately I use a MUA
> that deals with either way so I'm pretty indifferent.
>
> I don't have karma to change that setting, though.  Jeff?

I'm using GMail for this list at the moment. I'll have a look for
Greasemonkey stuff to work around my habit of just hitting reply to
list emails, otherwise, I'll just have to be more careful!

>
> Cheers
>
>         Stefan
>

An initial draft of the patch is done; I'm just writing more tests
around some serialization, so I'll create a patch on sourceforge
hopefully this weekend.

Cheers,

James

Re: [Xmlunit-general] Schema Validation patch development

[James A. <jam...@gm...>]

Initial patch up on sourceforge. There are still a few bits I'd like
to improve, but I was hoping for some peer review.

http://sourceforge.net/tracker/index.php?func=detail&aid=1630204&group_id=23187&atid=377770

The usage of the new code is expected in the new tests added to
test_Validator. I think that maybe the setSchemaLocation method
belongs on the XMLUnit class, and that should expose a static method
for accessing them (using xml-commons resolver or a HashMap
internally) but I'm haven't had a chance to look at xml-commons
resolver yet to see whether that feels like a good idea.

Comments welcome.

Cheers,

James

On 05/01/07, James Abley <jam...@gm...> wrote:
> On 05/01/07, Stefan Bodewig <bo...@ap...> wrote:
> > On Tue, 2 Jan 2007, James Abley <jam...@gm...> wrote:
> > > On 22/12/06, Stefan Bodewig <bo...@ap...> wrote:
> > >> On Thu, 21 Dec 2006, James Abley <jam...@gm...> wrote:
> > >>
> > >> > I'm writing a patch for consideration that will handle schema
> > >> > validation.
> > >>
> > >> [please use CVS diff to create patches, it is easier to see what
> > >> has changed that way]
> > >
> > > Will do - I'm just TDDing the feature at the moment and wanted to
> > > give a little context.
> >
> > That's fine.  Thank you.
> >
> > >> > In developing the tests for this patch though, I seem to have hit
> > >> > a bug, and was hoping someone could help me see what I'm doing
> > >> > wrong.  The below tests fail, although I don't think they
> > >> > should. I can't see what properties I should be setting that
> > >> > would cause it to pass.
> > >>
> > >> >     private String expected = "<root
> > >> >     xmlns='http://www.example.com/xmlns/example'" + "
> > >> >         + xmlns:xsi='http://exampe.com/xmlns/not/schema-instance/namespace'/>";;
> > >> >     private String actual =  "<root xmlns='http://www.example.com/xmlns/example' />";
> > >>
> > >> Why do you expect them to be different?  The
> > >> http://exampe.com/xmlns/not/schema-instance/namespace is never
> > >> used.
> > >>
> > >
> > > Well, because it has the namespace declaration and to my current way
> > > of thinking, that's just an attribute with special semantics.
> >
> > The way XMLUnit treats it it is not an attribute at all.  This has a
> > few benefits like allowing to documents to be equal if they use the
> > same namespaces but chose different prefixes for example.
> >
> > If you declare a namespace but never use it, this will never be seen
> > by XMLUnit since (as you've seen) this doesn't affect the DOM tree at
> > all.
> >
> > Do you think this is a problem?
>
> It is a little annoying when I'm trying to write comprehensive tests.
> I've got around it by adding a flag to XMLUnit that prevents any xmlns
> attributes from being ignored, but I would probably want to do that in
> a more robust fashion as you suggest. If namespaces are being
> compared, then  (probably) the namespace name is the important thing
> to compare, rather than the prefixes. I'll write some tests around
> this area and see how it goes (in a separate patch though).
>
> >
> > > I just realised that the mails on this list don't have the
> > > Reply-To header set to the mailing list, hence the reason I've been
> > > replying to individuals rather than the list. Might it be possible
> > > to alter that?
> >
> > I'll note that setting ot not setting Reply-To headers is the subject
> > of heated discussions (just do a search for "reply-to considered
> > harmful" with your favorite search engine).  Fortunately I use a MUA
> > that deals with either way so I'm pretty indifferent.
> >
> > I don't have karma to change that setting, though.  Jeff?
>
> I'm using GMail for this list at the moment. I'll have a look for
> Greasemonkey stuff to work around my habit of just hitting reply to
> list emails, otherwise, I'll just have to be more careful!
>
> >
> > Cheers
> >
> >         Stefan
> >
>
> An initial draft of the patch is done; I'm just writing more tests
> around some serialization, so I'll create a patch on sourceforge
> hopefully this weekend.
>
> Cheers,
>
> James
>

Re: [Xmlunit-general] Schema Validation patch development

[James A. <jam...@gm...>]

On 08/01/07, James Abley <jam...@gm...> wrote:
> Initial patch up on sourceforge. There are still a few bits I'd like
> to improve, but I was hoping for some peer review.
>
> http://sourceforge.net/tracker/index.php?func=detail&aid=1630204&group_id=23187&atid=377770
>
> The usage of the new code is expected in the new tests added to

That meant to say "The usage of the new code is demonstrated by", but
I had just put down a paper on JMock
(http://www.mockobjects.com/files/evolving_an_edsl.ooplsa2006.pdf), so
there was a little disconnect between what I was writing and thinking
about!

> test_Validator. I think that maybe the setSchemaLocation method
> belongs on the XMLUnit class, and that should expose a static method
> for accessing them (using xml-commons resolver or a HashMap
> internally) but I'm haven't had a chance to look at xml-commons
> resolver yet to see whether that feels like a good idea.
>
> Comments welcome.
>
> Cheers,
>
> James
>
> On 05/01/07, James Abley <jam...@gm...> wrote:
> > On 05/01/07, Stefan Bodewig <bo...@ap...> wrote:
> > > On Tue, 2 Jan 2007, James Abley <jam...@gm...> wrote:
> > > > On 22/12/06, Stefan Bodewig <bo...@ap...> wrote:
> > > >> On Thu, 21 Dec 2006, James Abley <jam...@gm...> wrote:
> > > >>
> > > >> > I'm writing a patch for consideration that will handle schema
> > > >> > validation.
> > > >>
> > > >> [please use CVS diff to create patches, it is easier to see what
> > > >> has changed that way]
> > > >
> > > > Will do - I'm just TDDing the feature at the moment and wanted to
> > > > give a little context.
> > >
> > > That's fine.  Thank you.
> > >
> > > >> > In developing the tests for this patch though, I seem to have hit
> > > >> > a bug, and was hoping someone could help me see what I'm doing
> > > >> > wrong.  The below tests fail, although I don't think they
> > > >> > should. I can't see what properties I should be setting that
> > > >> > would cause it to pass.
> > > >>
> > > >> >     private String expected = "<root
> > > >> >     xmlns='http://www.example.com/xmlns/example'" + "
> > > >> >         + xmlns:xsi='http://exampe.com/xmlns/not/schema-instance/namespace'/>";;
> > > >> >     private String actual =  "<root xmlns='http://www.example.com/xmlns/example' />";
> > > >>
> > > >> Why do you expect them to be different?  The
> > > >> http://exampe.com/xmlns/not/schema-instance/namespace is never
> > > >> used.
> > > >>
> > > >
> > > > Well, because it has the namespace declaration and to my current way
> > > > of thinking, that's just an attribute with special semantics.
> > >
> > > The way XMLUnit treats it it is not an attribute at all.  This has a
> > > few benefits like allowing to documents to be equal if they use the
> > > same namespaces but chose different prefixes for example.
> > >
> > > If you declare a namespace but never use it, this will never be seen
> > > by XMLUnit since (as you've seen) this doesn't affect the DOM tree at
> > > all.
> > >
> > > Do you think this is a problem?
> >
> > It is a little annoying when I'm trying to write comprehensive tests.
> > I've got around it by adding a flag to XMLUnit that prevents any xmlns
> > attributes from being ignored, but I would probably want to do that in
> > a more robust fashion as you suggest. If namespaces are being
> > compared, then  (probably) the namespace name is the important thing
> > to compare, rather than the prefixes. I'll write some tests around
> > this area and see how it goes (in a separate patch though).
> >
> > >
> > > > I just realised that the mails on this list don't have the
> > > > Reply-To header set to the mailing list, hence the reason I've been
> > > > replying to individuals rather than the list. Might it be possible
> > > > to alter that?
> > >
> > > I'll note that setting ot not setting Reply-To headers is the subject
> > > of heated discussions (just do a search for "reply-to considered
> > > harmful" with your favorite search engine).  Fortunately I use a MUA
> > > that deals with either way so I'm pretty indifferent.
> > >
> > > I don't have karma to change that setting, though.  Jeff?
> >
> > I'm using GMail for this list at the moment. I'll have a look for
> > Greasemonkey stuff to work around my habit of just hitting reply to
> > list emails, otherwise, I'll just have to be more careful!
> >
> > >
> > > Cheers
> > >
> > >         Stefan
> > >
> >
> > An initial draft of the patch is done; I'm just writing more tests
> > around some serialization, so I'll create a patch on sourceforge
> > hopefully this weekend.
> >
> > Cheers,
> >
> > James
> >
>