I wrote a perl version of 'nco_tst.sh' because I do not like having to scan through all the output it generates to determine whether things are working. It is called 'nco_tst.pl' and currently functions the same as nco_tst.sh except that tests that succeed are not output. A summary is given at the end. It currently does not deviate much from 'nco_tst.sh' because I wrote it by simply doing a bunch of regex replacements of the original script into perl syntax. to perform only certain tests, list them on the command line, i.e.
rorik$ ./nco_tst.pl ncks ncap
I did take the liberty to change a couple things, but I'm not sure they were legal. For example, if a test is suppose to fail, like one of the ncwa tests, the expected result is the error message, so it gets filtered out as a success. I filtered the ncap test the same way. I also changed the print format on two tests from "%f" to "%3.2f" to make them match.
There is currently one test that fails, that could be fixed with a different print format also I believe. A 'verbose' option could be added to print all the test regardless of success for failure if that would be useful.
If you look at the script, adding new tests is pretty straightforward: list each test in the @test array, add a description of the test in $description, and the expected result is $expected, which can be either numeric or a string (which is how error messages can be the correct result.)
Let me know if this seems helpful.
rorik
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> I wrote a perl version of 'nco_tst.sh' because I do not like having to scan
> through all the output it generates to determine whether things are working.
Great!
> It is called 'nco_tst.pl' and currently functions the same as nco_tst.sh except
> that tests that succeed are not output. A summary is given at the end. It
> currently does not deviate much from 'nco_tst.sh' because I wrote it by simply
> doing a bunch of regex replacements of the original script into perl syntax.
> to perform only certain tests, list them on the command line, i.e.
>
> rorik$ ./nco_tst.pl ncks ncap
>
> I did take the liberty to change a couple things, but I'm not sure they were
> legal. For example, if a test is suppose to fail, like one of the ncwa tests,
> the expected result is the error message, so it gets filtered out as a success.
> I filtered the ncap test the same way. I also changed the print format on two
> tests from "%f" to "%3.2f" to make them match.
Both seem to be appropriate changes
> There is currently one test that fails, that could be fixed with a different
> print format also I believe.
Yes, I changed the print format to %g and now it passes.
I also changed the expected on the test where the exact answer did
not matter, so now there are no failures on my Linux box
> A 'verbose' option could be added to print all
> the test regardless of success for failure if that would be useful.
Yes, these both sound like good ideas.
Also it would be helpful to document the invocation options at the top
of the file or when it's invoked with, e.g., -help.
Another improvement would be to assign an unambiguous number or string
to each test.
Another improvement would be to have it die on startup if it can't
find in.nc, since some users might get confused.
Also the ncap testing remains rather crude, in that the long testing
script ncap.in should be merged into nco_tst.pl, and ncap is very
lenient about ignoring errors and continuing on despite them.
> If you look at the script, adding new tests is pretty straightforward: list
> each test in the @test array, add a description of the test in $description,
> and the expected result is $expected, which can be either numeric or a string
> (which is how error messages can be the correct result.)
Yes, it's much more modular
> Let me know if this seems helpful.
It makes regression testing much easier! Let's adopt it and drop nco_tst.sh.
I just deleted nco_tst.sh, and replaced it everywhere I could think of
with nco_tst.pl.
Thanks!
Charlie
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I wrote a perl version of 'nco_tst.sh' because I do not like having to scan through all the output it generates to determine whether things are working. It is called 'nco_tst.pl' and currently functions the same as nco_tst.sh except that tests that succeed are not output. A summary is given at the end. It currently does not deviate much from 'nco_tst.sh' because I wrote it by simply doing a bunch of regex replacements of the original script into perl syntax. to perform only certain tests, list them on the command line, i.e.
rorik$ ./nco_tst.pl ncks ncap
I did take the liberty to change a couple things, but I'm not sure they were legal. For example, if a test is suppose to fail, like one of the ncwa tests, the expected result is the error message, so it gets filtered out as a success. I filtered the ncap test the same way. I also changed the print format on two tests from "%f" to "%3.2f" to make them match.
There is currently one test that fails, that could be fixed with a different print format also I believe. A 'verbose' option could be added to print all the test regardless of success for failure if that would be useful.
If you look at the script, adding new tests is pretty straightforward: list each test in the @test array, add a description of the test in $description, and the expected result is $expected, which can be either numeric or a string (which is how error messages can be the correct result.)
Let me know if this seems helpful.
rorik
Hi Rorik,
> I wrote a perl version of 'nco_tst.sh' because I do not like having to scan
> through all the output it generates to determine whether things are working.
Great!
> It is called 'nco_tst.pl' and currently functions the same as nco_tst.sh except
> that tests that succeed are not output. A summary is given at the end. It
> currently does not deviate much from 'nco_tst.sh' because I wrote it by simply
> doing a bunch of regex replacements of the original script into perl syntax.
> to perform only certain tests, list them on the command line, i.e.
>
> rorik$ ./nco_tst.pl ncks ncap
>
> I did take the liberty to change a couple things, but I'm not sure they were
> legal. For example, if a test is suppose to fail, like one of the ncwa tests,
> the expected result is the error message, so it gets filtered out as a success.
> I filtered the ncap test the same way. I also changed the print format on two
> tests from "%f" to "%3.2f" to make them match.
Both seem to be appropriate changes
> There is currently one test that fails, that could be fixed with a different
> print format also I believe.
Yes, I changed the print format to %g and now it passes.
I also changed the expected on the test where the exact answer did
not matter, so now there are no failures on my Linux box
> A 'verbose' option could be added to print all
> the test regardless of success for failure if that would be useful.
Yes, these both sound like good ideas.
Also it would be helpful to document the invocation options at the top
of the file or when it's invoked with, e.g., -help.
Another improvement would be to assign an unambiguous number or string
to each test.
Another improvement would be to have it die on startup if it can't
find in.nc, since some users might get confused.
Also the ncap testing remains rather crude, in that the long testing
script ncap.in should be merged into nco_tst.pl, and ncap is very
lenient about ignoring errors and continuing on despite them.
> If you look at the script, adding new tests is pretty straightforward: list
> each test in the @test array, add a description of the test in $description,
> and the expected result is $expected, which can be either numeric or a string
> (which is how error messages can be the correct result.)
Yes, it's much more modular
> Let me know if this seems helpful.
It makes regression testing much easier! Let's adopt it and drop nco_tst.sh.
I just deleted nco_tst.sh, and replaced it everywhere I could think of
with nco_tst.pl.
Thanks!
Charlie
Nice one Rorik