This has already been implemented, was cleaning out my mailbox and thought this mail thread would be useful and copied it over.
-sujit
>>From: James CE Johnson
>>To: spal@users.sourceforge.net
>>Subject: sqlunit and ant
>>Date: Wed, 27 Aug 2003 10:45:16 -0400
>>
>>Hi Sujit,
>>
>>Shouldn't the ant process fail if a sqlunit testcase fails?
>>
>>Thanks,
>>James
>>
>>
>
>
-----
> Hi James,
>
> I thought it made more sense for the process to continue and do all the
> stored procedures, since a failure in one stored procedure does not
> necessarily mean that all the other ones down the line are bad. Although
> this does conflict with Ant's behavior, and may confuse SQLUnit users
> who are familiar with JUnit. I think its documented but I may be wrong,
> in any case, I will update the docs to specify this behavior. Thanks
> for pointing it out.
>
> Do you see situations where this assumption would be incorrect? if so,
> let me know, and we could probably make this behavior configurable.
>
> - sujit
>
---
From: "James CE Johnson"
Subject: Re: sqlunit and ant
Date: Thu, 28 Aug 2003 13:31:43 -0400 (EDT)
Hi Sujit,
Thanks for the feedback.
We use an autobuilder (anthill) which pulls our source any time it
changes, builds the entire app, runs the unit tests, deploys the app, runs
the integration tests, etc... You get the idea. I would like to use
sqlunit so that we can test our data integrity and stored procs in this
same process. However, if the build succeeds when the tests fail we never
get notified of the failures.
I like the fact that you report failures then continue to execute the
other tests. What would be cool would be to throw a BuildException after
all of the tests have been run if the sqlunit task has a 'failOnFailures'
type attribute set.
Thanks again,
James
---
Sujit Pal wrote:
Hi James,
Thats a very cool idea. I looked at the code since I read your email last night. I did make the change you suggested and found that even if there is a test failure and SQLUnit task throws the BuildException, ant still says Build OK. Havent figured out why yet...
Anyway, I was thinking about this and I think that SQLUnit /should/ throw a BuildException on a test failure regardless of any configuration. It should not matter either way to people like me who run it as a console app, but it is important to people like you who want to run it within a batch process. However, what IMO /should/ be configurable is the ability to stop testing after a failure. So if your test was insert a record, update, select, and it failed on the first one, there is no point testing the other ones. So I am suggesting the following change:
1) BuildException should be thrown in case of a failure of one or more tests in the suite.
2) If a sqlunit attribute failonerror is set, then SQLUnit should throw a BuildException the first time it encounters a test failure and stop processing.
I am going to try to do a release with these changes over the long weekend, but failing that, it should be in by mid next week or so. I'll send you a mail when I make the release or you can register yourself as a listener on the site.
Hope this works for you,
Sujit
---
Hi Sujit,
That sounds like an excellent idea!
I've built some stuff on top of junit also and had to solve the same 'make lots of noise on failure" issue somewhere along the way. I've not looked closely at your run() method but here's mine in case it might help.
public void run(TestResult result)
{
this.initialize();
assertNotNull("Initializer was not able to initalize wats.", this.wats);
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
PrintStream stream = new PrintStream(bytes);
This has already been implemented, was cleaning out my mailbox and thought this mail thread would be useful and copied it over.
-sujit
>>From: James CE Johnson
>>To: spal@users.sourceforge.net
>>Subject: sqlunit and ant
>>Date: Wed, 27 Aug 2003 10:45:16 -0400
>>
>>Hi Sujit,
>>
>>Shouldn't the ant process fail if a sqlunit testcase fails?
>>
>>Thanks,
>>James
>>
>>
>
>
-----
> Hi James,
>
> I thought it made more sense for the process to continue and do all the
> stored procedures, since a failure in one stored procedure does not
> necessarily mean that all the other ones down the line are bad. Although
> this does conflict with Ant's behavior, and may confuse SQLUnit users
> who are familiar with JUnit. I think its documented but I may be wrong,
> in any case, I will update the docs to specify this behavior. Thanks
> for pointing it out.
>
> Do you see situations where this assumption would be incorrect? if so,
> let me know, and we could probably make this behavior configurable.
>
> - sujit
>
---
From: "James CE Johnson"
Subject: Re: sqlunit and ant
Date: Thu, 28 Aug 2003 13:31:43 -0400 (EDT)
Hi Sujit,
Thanks for the feedback.
We use an autobuilder (anthill) which pulls our source any time it
changes, builds the entire app, runs the unit tests, deploys the app, runs
the integration tests, etc... You get the idea. I would like to use
sqlunit so that we can test our data integrity and stored procs in this
same process. However, if the build succeeds when the tests fail we never
get notified of the failures.
I like the fact that you report failures then continue to execute the
other tests. What would be cool would be to throw a BuildException after
all of the tests have been run if the sqlunit task has a 'failOnFailures'
type attribute set.
Thanks again,
James
---
Sujit Pal wrote:
Hi James,
Thats a very cool idea. I looked at the code since I read your email last night. I did make the change you suggested and found that even if there is a test failure and SQLUnit task throws the BuildException, ant still says Build OK. Havent figured out why yet...
Anyway, I was thinking about this and I think that SQLUnit /should/ throw a BuildException on a test failure regardless of any configuration. It should not matter either way to people like me who run it as a console app, but it is important to people like you who want to run it within a batch process. However, what IMO /should/ be configurable is the ability to stop testing after a failure. So if your test was insert a record, update, select, and it failed on the first one, there is no point testing the other ones. So I am suggesting the following change:
1) BuildException should be thrown in case of a failure of one or more tests in the suite.
2) If a sqlunit attribute failonerror is set, then SQLUnit should throw a BuildException the first time it encounters a test failure and stop processing.
I am going to try to do a release with these changes over the long weekend, but failing that, it should be in by mid next week or so. I'll send you a mail when I make the release or you can register yourself as a listener on the site.
Hope this works for you,
Sujit
---
Hi Sujit,
That sounds like an excellent idea!
I've built some stuff on top of junit also and had to solve the same 'make lots of noise on failure" issue somewhere along the way. I've not looked closely at your run() method but here's mine in case it might help.
public void run(TestResult result)
{
this.initialize();
assertNotNull("Initializer was not able to initalize wats.", this.wats);
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
PrintStream stream = new PrintStream(bytes);
result.startTest(this);
Collection suites = null;
if(Text.empty(this.wats.getDefaultSuite()))
{
suites = this.wats.getSuiteNames();
}
else
{ suites = new ArrayList(Arrays.asList(new String [] {this.wats.getDefaultSuite()}));
}
Iterator i = suites.iterator();
while (i.hasNext())
{ String suiteName = (String) i.next();
try
{ this.wats.executeSuite(suiteName, stream);
}
catch (AssertionFailedError e)
{ result.addFailure(this, e);
log.error(bytes.toString());
}
catch (Throwable e)
{ result.addError(this, e);
log.error(bytes.toString());
}
}
result.endTest(this);
}
Have a good weekend,
James