exist-development

[Exist-development] Junit test suite

[Dannes W. <da...@ex...>]

Folks,

I really think we should freeze the codebase at this moment and focus on the
stability of the junit test suite. I mean, we have a serious xmlrpc crash
(unclear where it is originated) and some other strange issues (here: 187
failures, 136 errors).

That can't be true!......

regards

Dannes

-- 
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

Re: [Exist-development] Junit test suite

[Wolfgang M. <wol...@ex...>]

I already spent more than a day to find out what causes those JVM
crashes in the test suite, but without any results. For example,
org.exist.xmlrpc.AllXmlRpcTests runs two tests: QuerySessionTest and
XmlRpcTest. If you run either of these on their own, they will run
through perfectly well. If you run them as a suite (AllXmlRpcTests),
the second one will crash. The issues are definitely related to jetty
somehow.

I would definitely be happy to get some help on this.

Wolfgang

Re: [Exist-development] Junit test suite

[Joe W. <jo...@gm...>]

Hi Dannes and Wolfgang,

> I really think we should freeze the codebase at this moment and focus on the
> stability of the junit test suite. I mean, we have a serious xmlrpc crash
> (unclear where it is originated) and some other strange issues (here: 187
> failures, 136 errors).
...
> I would definitely be happy to get some help on this.

I'm not sure how much I'd be able to help without java skills, but I'd
be happy to help in any way I can.  I just ran "./build.sh test" on
trunk and got 0 failures and 2 errors (as reported in
EXIST_HOME/test/junit/html/index.html).  The 2 errors are in
org.exist.security and org.exist.xmlrpc.

I'm admittedly a testing newbie here.  Is there a resource you could
point me to so I could understand junit or how eXist implements it?

Joe

Re: [Exist-development] Junit test suite

[Dannes W. <da...@ex...>]

Running on my Mac dev-box, I have 2 failures and 2 errors. Almost the same.
On the PC, with the big numbers I have also the Spatial index enabled.
Maybe there is a relation.... I remember Pierrick mentioned something in
this area...

D.

On Sat, Feb 6, 2010 at 8:00 PM, Joe Wicentowski <jo...@gm...> wrote:

> Hi Dannes and Wolfgang,
>
> > I really think we should freeze the codebase at this moment and focus on
> the
> > stability of the junit test suite. I mean, we have a serious xmlrpc crash
> > (unclear where it is originated) and some other strange issues (here: 187
> > failures, 136 errors).
> ...
> > I would definitely be happy to get some help on this.
>
> I'm not sure how much I'd be able to help without java skills, but I'd
> be happy to help in any way I can.  I just ran "./build.sh test" on
> trunk and got 0 failures and 2 errors (as reported in
> EXIST_HOME/test/junit/html/index.html).  The 2 errors are in
> org.exist.security and org.exist.xmlrpc.
>
> I'm admittedly a testing newbie here.  Is there a resource you could
> point me to so I could understand junit or how eXist implements it?
>
> Joe
>
>
> ------------------------------------------------------------------------------
> The Planet: dedicated and managed hosting, cloud storage, colocation
> Stay online with enterprise data centers and the best network in the
> business
> Choose flexible plans and management services without long-term contracts
> Personal 24x7 support from experience hosting pros just a phone call away.
> http://p.sf.net/sfu/theplanet-com
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>



-- 
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

Re: [Exist-development] Junit test suite

[Joe W. <jo...@gm...>]

Hi Dannes,

> Running on my Mac dev-box, I have 2 failures and 2 errors. Almost the same.
> On the PC, with the big numbers I have also the Spatial index enabled.
> Maybe there is a relation.... I remember Pierrick mentioned something in
> this area...

Ah, I should add that I was running on a Mac (10.6.2), without spatial
index.  I enabled all optional modules in conf.xml except those that
require build.properties or local.build.properties values to be
changed (JNDI, XSLFO, etc.).

One apparent bug in the teardown of a test: after the test was
complete, this entry remained in my /db/system/users.xml

        <user name="rudi" uid="3"
password="{MD5}Gh3JHJBzJcaScd3wyUS8cg=="
digest-password="0103d1b2d0c7691e111d7a40d0af50c9">
            <group>guest</group>
        </user>

Joe

Re: [Exist-development] Junit test suite

[Wolfgang M. <wol...@ex...>]

Hi Joe,

ok, so we are at least all getting those errors.

> I'm admittedly a testing newbie here.  Is there a resource you could
> point me to so I could understand junit or how eXist implements it?

We are mostly using junit4. This is probably rather difficult to
understand without knowing Java, though I recently tried to write more
tests in XQuery (see test/src/xquery/).

Anyway, each test class should be self-contained. If a class needs a
running server for its test, it will start one, usually in its
@BeforeClass method and stop it in @AfterClass. This worked perfectly
well until recently. But if you now try to run, for example,
AllXmlRpcTests:

bin/run.sh org.junit.runner.JUnitCore org.exist.xmlrpc.AllXmlRpcTests

you more or less see how the first test class completes and closes
down the server, but while the second test class starts up a new
jetty, Java just terminates without further notice.

Wolfgang

Re: [Exist-development] Junit test suite

[Joe W. <jo...@gm...>]

Hi Wolfgang,

> We are mostly using junit4. This is probably rather difficult to
> understand without knowing Java, though I recently tried to write more
> tests in XQuery (see test/src/xquery/).

Yeah, I see what you mean about the difficulty of understanding it
without knowing Java.  I did just have a look through the XQuery tests
in test/src/xquery, and found that pretty easy to understand.  Cool!
I didn't realize we had a framework for unit tests written in XQuery!
I've seen Chris Wallace's blog posts suggesting some ways to do such
tests, and I've been following Florent Georges and Jeni Tennison's
XSpec project on googlecode.

> Anyway, each test class should be self-contained. If a class needs a
> running server for its test, it will start one, usually in its
> @BeforeClass method and stop it in @AfterClass. This worked perfectly
> well until recently. But if you now try to run, for example,
> AllXmlRpcTests:
>
> bin/run.sh org.junit.runner.JUnitCore org.exist.xmlrpc.AllXmlRpcTests
>
> you more or less see how the first test class completes and closes
> down the server, but while the second test class starts up a new
> jetty, Java just terminates without further notice.

I see....  Hmm...  I think this gets into Java territory, above my
head.  I'm happy to run any tests that would be useful, though, and
report my results.

Joe

Re: [Exist-development] Junit test suite

[Dmitriy S. <sha...@gm...>]

Hello,

I did commit fix for AllXmlRpcTests
( http://exist.svn.sourceforge.net/exist/?rev=11135&view=rev ) , please
confirm that this one fixed.

If not we should make server @JettyStart static.

-- 
Cheers,

Dmitriy Shabanov

On Sat, 2010-02-06 at 20:23 +0100, Wolfgang Meier wrote:
> Hi Joe,
> 
> ok, so we are at least all getting those errors.
> 
> > I'm admittedly a testing newbie here.  Is there a resource you could
> > point me to so I could understand junit or how eXist implements it?
> 
> We are mostly using junit4. This is probably rather difficult to
> understand without knowing Java, though I recently tried to write more
> tests in XQuery (see test/src/xquery/).
> 
> Anyway, each test class should be self-contained. If a class needs a
> running server for its test, it will start one, usually in its
> @BeforeClass method and stop it in @AfterClass. This worked perfectly
> well until recently. But if you now try to run, for example,
> AllXmlRpcTests:
> 
> bin/run.sh org.junit.runner.JUnitCore org.exist.xmlrpc.AllXmlRpcTests
> 
> you more or less see how the first test class completes and closes
> down the server, but while the second test class starts up a new
> jetty, Java just terminates without further notice.
> 
> Wolfgang

Re: [Exist-development] Junit test suite

[Wolfgang M. <wol...@ex...>]

> I did commit fix for AllXmlRpcTests
> ( http://exist.svn.sourceforge.net/exist/?rev=11135&view=rev ) , please
> confirm that this one fixed.

Mmmh, I think this is a bad fix. Every test should be self-contained.
The server should be stopped when the test has finished, so the next
test can

Anyway, I believe I know where the problem is ....

Wolfgang