I am looking for a means in which I can run multiple test suites and combine the results into a cumulative output. For example, if I run two tests with results:
5 tests, 1 failure, 0 errors
10 tests 2 failures, 0 errors
I would like to generate the net result:
15 tests, 3 failures, 0 errors
Of course, the project I am considering using SQLUnit for will have many more tests suites then my example.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The XSLT that I developed should allow for what you want Sean.
The changes should be in the most recent code within SQLUnit (after 4.9 was released), so you will need to pull the necessary files from CVS.
Once you have these new files, you can use the XSLT to process the resulting XML into XHTML by using the following command:
ant -Doutput.file=MyOutputFile -Dlog.format=canoo sqlunit-nested -Dtestdir="PathToMyTestDirectory"
This will generate MyOutputFile.html in the output directory from MyOutputFile.xml (which is generated by the Canoo XML Reporter). Note that the output.file property does not specify an extension. The output file will be placed within the output directory (specified by the output.dir property).
Alternatively, you could modify your local copy of build.xml to automatically invoke the canoo2html target, set up your output file name, and the log format. This way you won't have to specify this information every time on the command line.
For all the gory details on the current state of this enhancement, check out the following thread:
The way I usually describe this XSL transform performs its calculations is in terms of "test suites", "test cases" and "tests".
A SQLUnit XML file is considered to be a "test case" for purposes of the XSLT calcuation. A test case is composed of one or more tests. Each test tag within a test case is counted as a test.
"Test suites" are analogous to folders or directories that contain test cases, or other test suites. The output from the XSL transform does not currently calculate statistics on test suites.
The Canoo2HTML XSL transform will provide the following information:
1. Test case summary (total executed/pass/fail)
2. Test Summary (total executed/pass/fail)
3. Test Case breakdown (total tests/pass/fail) for each test case
In addition, the start of the HTML report will indicate Success in green and Failure in red.
If there are one or more failures, these are indicated within the test failure details. These details are extract directly from the Canoo XML output (so you will need to get used to each error, what it means, and how to correct it.)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I am looking for a means in which I can run multiple test suites and combine the results into a cumulative output. For example, if I run two tests with results:
5 tests, 1 failure, 0 errors
10 tests 2 failures, 0 errors
I would like to generate the net result:
15 tests, 3 failures, 0 errors
Of course, the project I am considering using SQLUnit for will have many more tests suites then my example.
Hi Sean,
Take a look at the tool James Henderson contributed, which allows you to build up results with these stats in an HTML page.
James would be the best person to answer questions about this, so I'll defer to him.
-sujit
Hi Sean, Sujit,
I apologize for not responding sooner.
The XSLT that I developed should allow for what you want Sean.
The changes should be in the most recent code within SQLUnit (after 4.9 was released), so you will need to pull the necessary files from CVS.
Once you have these new files, you can use the XSLT to process the resulting XML into XHTML by using the following command:
ant -Doutput.file=MyOutputFile -Dlog.format=canoo sqlunit-nested -Dtestdir="PathToMyTestDirectory"
This will generate MyOutputFile.html in the output directory from MyOutputFile.xml (which is generated by the Canoo XML Reporter). Note that the output.file property does not specify an extension. The output file will be placed within the output directory (specified by the output.dir property).
Alternatively, you could modify your local copy of build.xml to automatically invoke the canoo2html target, set up your output file name, and the log format. This way you won't have to specify this information every time on the command line.
For all the gory details on the current state of this enhancement, check out the following thread:
http://sourceforge.net/forum/forum.php?thread_id=1295259&forum_id=265576
I have been busy lately and have not had a chance to follow up to Sujit's last post in the thread.
Hope this helps.
-jh
One other thing
You will need to replace the ampersand entity (i.e., &) with a true ampersand in my previous post in order to link to the thread.
-jh
Oh, yeah, one other thing...
The way I usually describe this XSL transform performs its calculations is in terms of "test suites", "test cases" and "tests".
A SQLUnit XML file is considered to be a "test case" for purposes of the XSLT calcuation. A test case is composed of one or more tests. Each test tag within a test case is counted as a test.
"Test suites" are analogous to folders or directories that contain test cases, or other test suites. The output from the XSL transform does not currently calculate statistics on test suites.
The Canoo2HTML XSL transform will provide the following information:
1. Test case summary (total executed/pass/fail)
2. Test Summary (total executed/pass/fail)
3. Test Case breakdown (total tests/pass/fail) for each test case
In addition, the start of the HTML report will indicate Success in green and Failure in red.
If there are one or more failures, these are indicated within the test failure details. These details are extract directly from the Canoo XML output (so you will need to get used to each error, what it means, and how to correct it.)