From: Jason S. <jas...@gm...> - 2006-01-27 07:29:17
|
Hey William, Just to keep in mind here's the goal: * make it easy to write Perl test scripts - so that we can quickly find when the Perl module breaks due to code changes * make it easy for testers to find any breakage - so they don't have to pour over pages of output * write thorough tests - so that all testable features are covered On 1/27/06, William S Fulton <ws...@fu...> wrote: > Jason Stewart wrote: > > Hey William, > > > > Sorry to be creating more work for you. > > > I don't mind a bit more work if it results in progress and am rather > pleased that someone is taking more interest on the perl side of things > as the perl module is also a bit neglected. Fair enough. > I can see that good testing tool yields benefits and using an industry > standard one is a good thing. If we can solve the inflexibility of the > tool then I think we have moved forward overall. I'm surprised there > isn't more control over the stdout/stderr. Do you think it will be > possible to modify it to get more control over the stdout/stderr to make > it quiet? While possible, I feel that modifying Test::More is the wrong level. Please excuse the long explanation - I know you're busy. The real issue is that SWIG doesn't have a dedicated test runner - instead it uses a bunch of make macros - and unless you're a big shell script fan, make sucks. Test::More is not a test runner - it is just a library of methods. Those methods help test writers create test scripts that have a strictly defined output to STDOUT/STDERR that can be easily parsed by Perl's built-in test runner: Test::Harness. Test::Harness provides methods to run an entire test-suite and parses the output of possibly hundreds of test scripts each with possibly hundreds of tests. Because it has to parse the STDOUT/STDERR of each script - that format has to be very well defined. Test::Harness is a fairly basic test-runner that prints a summary of all tests to STDOUT. The idea being that a tester can run any of the failed tests by hand to see all the verbose Test::More output for that test. That's the standard way testing is done in Perl. But SWIG's need is a bit different. Test::Harness has a different test model than SWIG - first compile all the files, then if and only if compilation succeeds run all the test scripts and print out a summary. What SWIG wants is to compile one file at a time, and run one test script at a time and don't print anything on success - very different model. Using Test::Harness as a test runner would add nothing except handling the verbose output of Test::More. So I chose to write my own test runner instead. I don't consider the current test runner script a hack - it is a simple test runner that takes test output and formats it the way we require. Changing the output format of Test::More would be much more work, and it would have to be redone if we every wanted to update the version distributed with SWIG. > I'd imagine it is the sort of thing that would be accepted > upstream by the maintainer. I really don't think so - the maintainer would probably say: Test::More is not a test runner - Test::Harness is, Test::More is a library for writing Perl tests designed to be parsed by Test::Harness. I hope this makes things more clear. Let me know what you think is best. Cheers, jas. |