[Perlunit-devel] Assertions for strings and nested data structures

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Well, I have played around a bit with assert_str_equals and assert_deep_equals as implemented in Assert.pm, and I have to say I much prefer my homegrown assert_string_equals and assert_structures_are_equal methods (which I implemented in a subclass TestCaseWithHelpers of TestCase).

Attached are two TestCases which perform the same two assertions, one on a string, and one on a nested data structure, using either the Assert.pm methods or the methods I developed in my TestCaseWithHelpers.

If I run the AssertTest.pm file, the first thing I notice is that it only reports the failure for the test_assert_str_equals() test. In fact, it's even worse than this. If I include this AssertTest in a suite that includes other testcases, the testing process stops as soon as it fails in the AssertTest::test_assert_str_equals() test.

In contrast, if I run the AlainAssert.pm test case, it reports both failures from test_assert_string_equals() and test_assert_structures_are_equal(). Maybe I'm not using Assert.pm properly? I based my code on an example from this pages: http://search.cpan.org/~dexter/Test-Assert-0.03/lib/Test/Assert.pm.

The second thing I notice is that Assert.pm's assert_str_equals() only tells me that the two strings differ, without showing me where they actually differ:

===
.Exception::Assertion: Expected 'To be, or not to be, that is the question:
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles,
And by opposing end them?', got 'To be, or not to be, that is the question:
Whether tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles,
And by opposing end them?' at C:\Users\Desiletsa\Documents\eclipse_workspace\WeBiText/UnitTests/AssertTest.pm line 28
===

In contrast, if I run AlainAssert.pm, I get:

===
2) ../../IIPerlUtils/TestingAndDebugging/TestCaseWithHelpers.pm:143 - test_assert_string_equals(AlainAssertTest)

Strings differed. Position of first found difference is shown below.

*** Got string: ***
'To be, or not to be, that is the question:
Whether <*** DIFF FOUND HERE ***>'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles,
And by opposing end them?'

*** Expected string: ***
'To be, or not to be, that is the question:
Whether <*** DIFF FOUND HERE ***>tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles,
And by opposing end them?'
===

Which immediately tells me that the culprit is the missing single quote in front of "'tis nobler".

The third thing I notice is that if I comment out the test test_assert_str_equals() in AssertTest, so that I can see the failure for the other test test_assert_deep_equal(), I get this:

===
.Exception::Assertion: Nevermind: Structures begin differing at:
  $a->[1]{gender} = 'M'
  $b->[1]{gender} = 'F' at C:\Users\Desiletsa\Documents\eclipse_workspace\WeBiTe
xt/UnitTests/AssertTest.pm line 43
etc..
===

This is pretty good, in that it tells me that there is a problem with the gender of one of the entries in the structures that are being compared. But I have to do a bit of mental work to figure out which of the two male entries it is referring to (namely, the second entry, 'Dave Bloke'). 

In contrast, if I run test AlainAssert, I get:

===
1) ../../IIPerlUtils/TestingAndDebugging/TestCaseWithHelpers.pm:143 - test_assert_structures_are_equal(AlainAssertTest)
Nevermind
The two data structures differed. Below is a diff between the serialization of those data structures.

Strings differed. Position of first found difference is shown below.

*** Got string: ***
':  [ # ARRAY
:  :  { # HASH
:  :  :  first_name => Bloe
:  :  :  gender => M
:  :  :  last_name => Joe
:  :  }
:  :  { # HASH
:  :  :  first_name => Bloke
:  :  :  gender => <*** DIFF FOUND HERE ***>M
:  :  :  last_name => Dave
:  :  }
:  :  { # HASH
:  :  :  first_name => Doe
:  :  :  gender => F
:  :  :  last_name => Jane
:  :  }
:  ]
'

*** Expected string: ***
':  [ # ARRAY
:  :  { # HASH
:  :  :  first_name => Bloe
:  :  :  gender => M
:  :  :  last_name => Joe
:  :  }
:  :  { # HASH
:  :  :  first_name => Bloke
:  :  :  gender => <*** DIFF FOUND HERE ***>F
:  :  :  last_name => Dave
:  :  }
:  :  { # HASH
:  :  :  first_name => Doe
:  :  :  gender => F
:  :  :  last_name => Jane
:  :  }
:  ]
'
===

This output makes it immeediatly clear that the problem is with the gender of Dave Bloke, not Joe Bloe.

Mind you, I can see that in some circumstances, knowing that the problem is with the 2nd entry may be more important than knowing the problem is with Dave Bloke. So I think the best thing would be a mix of the two outputs, namely something like this:

1) ../../IIPerlUtils/TestingAndDebugging/TestCaseWithHelpers.pm:143 - test_assert_structures_are_equal(AlainAssertTest)

The two data structures differed. 
  $a->[1]{gender} = 'M'
  $b->[1]{gender} = 'F'

Below is a diff between the serialization of those data structures.

*** Got string: ***
':  [ # ARRAY
:  :  { # HASH
:  :  :  first_name => Bloe
:  :  :  gender => M
:  :  :  last_name => Joe
:  :  }
:  :  { # HASH
:  :  :  first_name => Bloke
:  :  :  gender => <*** DIFF FOUND HERE ***>M
:  :  :  last_name => Dave
:  :  }
:  :  { # HASH
:  :  :  first_name => Doe
:  :  :  gender => F
:  :  :  last_name => Jane
:  :  }
:  ]
'

*** Expected string: ***
':  [ # ARRAY
:  :  { # HASH
:  :  :  first_name => Bloe
:  :  :  gender => M
:  :  :  last_name => Joe
:  :  }
:  :  { # HASH
:  :  :  first_name => Bloke
:  :  :  gender => <*** DIFF FOUND HERE ***>F
:  :  :  last_name => Dave
:  :  }
:  :  { # HASH
:  :  :  first_name => Doe
:  :  :  gender => F
:  :  :  last_name => Jane
:  :  }
:  ]
'
===

Another thing I dislike about the Assert.pm version is that it prints a long call stack that looks like this, all of which is useless, except the first line. 

===
at C:\Users\Desiletsa\Documents\eclipse_workspace\WeBiText/UnitTests/AssertTest.pm line 43
        $_ = AssertTest::test_assert_deep_equal("AssertTest=HASH(0x357621c)") ca
lled in package Test::Unit::TestCase at C:/Perl/site/lib/Test/Unit/TestCase.pm l
ine 75
        $_ = Test::Unit::TestCase::run_test("AssertTest=HASH(0x357621c)") called
 in package Test::Unit::TestCase at C:/Perl/site/lib/Test/Unit/TestCase.pm line
61
        $_ = Test::Unit::TestCase::__ANON__() called in package Error::subs at C
:/Perl/site/lib/Error.pm line 408
        $_ = eval {...} called in package Error::subs at C:/Perl/site/lib/Error.
pm line 407
        $_ = Error::subs::try("CODE(0x39d63ac)", "HASH(0x39d648c)") called in pa
ckage Test::Unit::TestCase at C:/Perl/site/lib/Test/Unit/TestCase.pm line 66
        $_ = Test::Unit::TestCase::run_bare("AssertTest=HASH(0x357621c)") called
 in package Test::Unit::Result at C:/Perl/site/lib/Test/Unit/Result.pm line 103
        $_ = Test::Unit::Result::__ANON__() called in package Test::Unit::Result
 at C:/Perl/site/lib/Test/Unit/Result.pm line 119
        $_ = Test::Unit::Result::__ANON__() called in package Error::subs at C:/
Perl/site/lib/Error.pm line 415
        $_ = eval {...} called in package Error::subs at C:/Perl/site/lib/Error.
pm line 407
        $_ = Error::subs::try("CODE(0x39d626c)", "HASH(0x39d5e3c)") called in pa
ckage Test::Unit::Result at C:/Perl/site/lib/Test/Unit/Result.pm line 133
        $_ = Test::Unit::Result::run_protected("Test::Unit::Result=HASH(0x39d5f9
c)", "AssertTest=HASH(0x357621c)", "CODE(0x39d5cac)") called in package Test::Un
it::Result at C:/Perl/site/lib/Test/Unit/Result.pm line 107
        $_ = Test::Unit::Result::run("Test::Unit::Result=HASH(0x39d5f9c)", "Asse
rtTest=HASH(0x357621c)") called in package Test::Unit::TestCase at C:/Perl/site/
lib/Test/Unit/TestCase.pm line 51
        $_ = Test::Unit::TestCase::run("AssertTest=HASH(0x357621c)", "Test::Unit
::Result=HASH(0x39d5f9c)", "Test::Unit::TestRunner=HASH(0x2ca16ec)") called in p
ackage Test::Unit::TestSuite at C:/Perl/site/lib/Test/Unit/TestSuite.pm line 278

        $_ = Test::Unit::TestSuite::run("Test::Unit::TestSuite=HASH(0x38d6754)",
 "Test::Unit::Result=HASH(0x39d5f9c)", "Test::Unit::TestRunner=HASH(0x2ca16ec)")
 called in package Test::Unit::TestSuite at C:/Perl/site/lib/Test/Unit/TestSuite
.pm line 278
        $_ = Test::Unit::TestSuite::run("AllTests=HASH(0x38d6484)", "Test::Unit:
:Result=HASH(0x39d5f9c)", "Test::Unit::TestRunner=HASH(0x2ca16ec)") called in pa
ckage Test::Unit::TestRunner at C:/Perl/site/lib/Test/Unit/TestRunner.pm line 54

        $_ = Test::Unit::TestRunner::do_run("Test::Unit::TestRunner=HASH(0x2ca16
ec)", "AllTests=HASH(0x38d6484)", 0) called in package Test::Unit::TestRunner at
 C:/Perl/site/lib/Test/Unit/TestRunner.pm line 183
        $_ = Test::Unit::TestRunner::start("Test::Unit::TestRunner=HASH(0x2ca16e
c)", "AllTests") called in package main at run_tests.pl line 160
        ...propagated in package Error::subs at C:/Perl/site/lib/Error.pm line 4
34.
        ...propagated in package Error::subs at C:/Perl/site/lib/Error.pm line 4
34.
===

So, in conclusion, I will continue using my own homegrown assertions instead of the ones in Assert.pm. 

If people feel that it would make sense for those to be included in the standard version of TestCase.pm, I am more than happy to contribute them to the project, and to look after the integration. Otherwise, I will just post TestCaseWithHelpers somewhere, probably on my personal site:

http://alaindesilets.org/MyPublicSite/tiki-view_blog.php?blogId=1

Cheers,

Alain