Re: [Perlunit-users] 2 questions: traceback and running test suites and
Status: Beta
Brought to you by:
mca1001
Menu
▾
▴
Re: [Perlunit-users] 2 questions: traceback and running test suites and
|
From: Matthew A. <mc...@us...> - 2005-06-21 22:35:01
|
[Repost 'cos I got the wrong 'From' address. Hopefully the first one
will give up waiting for the moderator and go Away.]
On Tue, Jun 21, 2005 at 04:29:41PM -0400, Desilets, Alain wrote:
> I am a newbie to PerlUnit, but have used jUnit and pyUnit
> extensively.
Hello!
> First, is there a way to get PerlUnit to print a traceback (with the
> file name and line number of all the function and method calls in
> the call stack) when an exception is raised?
Um, I suspect there probably is, yes... but if I knew where the
setting was, I've forgotten. 8-(
Hmm, there was a thing about get_backtrace_on_fail. This was removed
a while back, it claims to be deprecated if you call it but it's
actually non-functional. I don't recall what the replacement or
outcome was.
Hmm, the "Error" base class contains
$Error::Debug = 0; # Generate verbose stack traces
at the top, so I've used that in the code at the bottom. I suspect
there's a tidy way to get this out, probably involves adding a
Listener to the TestRunner...? I've cheated by using Data::Dumper.
I have a more funky testrunner script which I normally use. I'll have
a poke about and add stacktrace info, erm, soon. I seem to have
completely forgotten my SF password. Again. Not a good sign for a
"maintainer".
> This is the default behaviour of both jUnit and pyUnit, but it seems
> PerlUnit only prints the message of the exception that was raised.
> This is sometimes enough to identify the source of the problem, but
> not if the error happens deep inside the system.
I know the problem. For some reason it doesn't trouble me much, if I
can figure out what trick it is I'm using, I'll explain.
Also in favour of short messages: it's good to have a concise
description of the problem, provided the detail is still available.
What would be really handy is having a place to dump extra data from
the test. Something built in to (a subclass of) the TestCase, so
stack traces are just a C-x C-f (file load) away, or a refresh of a
web page.
> Secondly, how do you run a suite of tests? I have tried 3 different
> approaches, and none give me exactly what I want. See sample code
> below with inlined comments.
> --- Sample Code:
>
> use strict;
>
> use Test::Unit::TestRunner;
> use Test::Unit::TestSuite;
> use Test::Unit::TestCase;
>
>
> package FooBar;
> our @ISA = qw(Test::Unit::TestCase);
>
> sub new {
> my ($class, @args) = @_;
> my $self = $class->SUPER::new(@args);
> return $self;
> }
As far as I'm aware, there should be no need to override 'new' for any
of the commonly used classes. You can just use the inherited one.
> sub test_fail {
> my ($self) = @_;
> $self->fail("failed in FooBar.");
> }
>
> package FooBar2;
> our @ISA = qw(Test::Unit::TestCase);
>
> sub new {
> my ($class, @args) = @_;
> my $self = $class->SUPER::new(@args);
> return $self;
> }
>
> sub test_fail {
> my ($self) = @_;
> $self->fail("failed in FooBar2.");
> }
Otherwise, looks OK so far...
> package MyTestSuite;
> use base qw(Test::Unit::TestSuite);
>
> sub new {
> my ($class) = @_;
> my $self = $class->SUPER::empty_new();
> no strict;
> $self->add_test(FooBar);
> $self->add_test(FooBar2);
> use strict;
> return $self;
> }
>
> sub name { 'My very own test suite' }
You can do this more easily with the first example in the
Test::Unit::TestSuite POD, just provide the name() as you've done plus
an include_tests() ,
sub include_tests {
return ( "FooBar", "FooBar2" );
}
or whatever shorthand you like for such things.
> package main;
>
> my $runner = Test::Unit::TestRunner->new();
>
> ##########################################################################
> # APPROACH 1: Implement subclass MyTestSuite of TestSuite, and feed its
> # name to TestRunner::start().
> # MyTestSuite will wrap a bunch of test cases into a suite.
> #
> # RESULT:
> # WORKS, BUT ALL ERRORS REPORTED AS THOUGH THEY WERE FROM THE SAME
> # TEST CASE
> #
> # I need to see errors from different TestCases, because I often have TestCases
> # that test different subclasses of a same root class, and these TestCases
> # use the same methods inherited from a common TestCase subclass.
> ##########################################################################
>
> #no strict;
> #$runner->start(MyTestSuite);
> #use strict;
That should work. I'm puzzled by the 'no strict' thing, you can just
pass the classname as a string...? But that's not relevant.
So I save it out and tried it,
mca1001@doorstop:/tmp$ perl mail.pl
.F.F
Time: 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU)
!!!FAILURES!!!
Test Results:
Run: 2, Failures: 2, Errors: 0
There were 2 failures:
1) mail.pl:19 - test_fail(FooBar)
failed in FooBar.
2) mail.pl:33 - test_fail(FooBar2)
failed in FooBar2.
Test was not successful.
which looks about right to me. Two test methods called test_fail, in
their respective classes.
Did the 'make test' run OK? (Apart from the broken tests)
What are you expecting to see different to this?
> ##########################################################################
> # APPROACH 2: Create an empty TestSuite, add TestCases to it and feed it to
> # TestRunner::start()
This would be equivalent to the subclassing approach, but for the fact
that the TestRunner is expecting a classname not an instance of the
TestSuite. You passed an instance of the suite to $runner->start ,
> # Couldn't load Test::Unit::TestSuite=HASH(0x18255a4) in any of the
> # supported ways at ../../../perlib//Test/Unit/Loader.pm line 68.
I vaguely remember doing this too... the clue is that it's trying to
load a class named "Test::Unit::TestSuite=HASH(0x18255a4)", and that's
the stringified object. I think it should either catch this case and
give a more helpful message, or cope with it.
The TestRunner will apparently also accept the names of files to pass
to Test::Unit::UnitHarness, in various forms.
> ##########################################################################
> # APPROACH 3: Run each TestCase using a different TestRunner.
You should only need the one TestRunner instance. That's the
convenience of having the TestSuite, pile it all in and let it run.
Hacking your code about to have some subclassing of the first
TestCase, (and almost no comments)
----8<----
use strict;
package FooBar;
use base 'Test::Unit::TestCase';
sub test_fail {
my ($self) = @_;
$self->fail("failed in FooBar's test_fail,\n \$self = '$self', an instance of ".ref($self));
}
package FooBar2;
use base 'FooBar';
sub test_pass { }
package Wibble3;
use base 'FooBar2';
package MyTestSuite;
use base qw(Test::Unit::TestSuite);
sub include_tests {qw{FooBar FooBar2 Wibble3}}
sub name { 'My very own test suite' }
package main;
use Test::Unit::TestRunner;
use Data::Dumper;
my $runner = Test::Unit::TestRunner->new();
$Error::Debug = 1;
$runner->start('MyTestSuite'); # Approach 1
print "-"x70,"\n", Data::Dumper->Dump([$runner], ['runner']);
----8<----
Tests being run will be
FooBar->test_fail
FooBar2->test_fail [inherited from FooBar]
FooBar2->test_pass
Wibble3->test_fail [inherited from FooBar]
Wibble3->test_pass [inherited from FooBar2]
and the output I get looks like this,
mca1001@doorstop:/tmp$ perl mail.pl
.F.F..F.
Time: 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU)
!!!FAILURES!!!
Test Results:
Run: 5, Failures: 3, Errors: 0
There were 3 failures:
1) mail.pl:8 - test_fail(FooBar)
failed in FooBar's test_fail,
$self = 'FooBar=HASH(0x8164260)', an instance of FooBar
2) mail.pl:8 - test_fail(FooBar2)
failed in FooBar's test_fail,
$self = 'FooBar2=HASH(0x81644e8)', an instance of FooBar2
3) mail.pl:8 - test_fail(Wibble3)
failed in FooBar's test_fail,
$self = 'Wibble3=HASH(0x81643a4)', an instance of Wibble3
Test was not successful.
which again looks right to me. After this, there follows a messy
printout of the internals of the TestRunner and your
Test::Unit::Failure instances, which do contain the trace info.
I set the failure method to explain what the $self is. The TestCases
follow the standard form of inheritance, that each of the subclasses
will see the test_fail method unless you override it.
Short answer is, I can't tell what it is that's broken in your first
approach, but I will say that this system isn't hugely forgiving if
you step off the expected usage path. Sorry about that.
Matthew #8-)
|
