Re: [Module::Build] [RFC] author tests

[Chris D. <ch...@cl...>]

On Feb 1, 2006, at 10:35 PM, Tyler MacDonald wrote:

> Chris Dolan <ch...@cl...> wrote:
>> There is a class of tests that module authors perform that end users
>> are not expected to run.  For example code coverage tests, spelling
>> tests, coding style tests, etc.  These tests are either prohibitively
>> expensive or complicated or unpredictable for end users to run.  I
>> call these "private tests" or author tests.
>
> 	I really like this idea. But as you pointed out, it's not just
> authors that need to worry about running these tests, it's packagers
> (ppm/deb/etc), automated testers (cpants/testers.cpan.org/etc), and  
> hackers.
> I'd suggest we call these "exhaustive" tests.

No, I disagree.  I'm specifically talking about author tests, NOT  
packager tests.  Things like Test::Spelling are pointless and  
difficult for packagers to execute because Test::Spelling relies on  
an external aspell or ispell program *and* performs differently in  
the presence of an author's custom dictionary (mine has "Dolan"; does  
yours?)

These specifically are not exhaustive tests but spit-and-polish tests.

To make this less abstract, let me list the specific author tests  
that I employ for most of my CPAN modules, along with an explanation  
of why it wouldn't work for a packager

  * copyright.t - Ensures that there is a "Copyright ".([localtime]-> 
[5]+1900) somewhere in every .pm file.  Will break 11 months from now.
  * distribution.t - Relies on Test::Distribution, which is not in my  
prereq list
  * perlcritic.t - Runs Test::Perl::Critic on all .pm files.  Will  
fail without my specific $HOME/.perlcriticrc and will fail with  
future, more exhaustive versions of P::C
  * spelling.t - Runs Test::Spelling.  Will fail without my custom  
dictionary
  * versionsync.t - Checks that the $VERSION is the same in all bin/*  
and *.pm files.  This test is pointless after release, since it's  
already been tested before release
  * pod.t - Checks POD validity.  This test is pointless after  
release, since it's already been tested before release
  * pod-coverage.t - Checks POD completeness.  This test is pointless  
after release, since it's already been tested before release

and one I have not yet employed:
  * coverage.t - Ensures that Devel::Cover totals are higher than  
some threshold

Chris
-- 
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf

Clotho Advanced Media, Inc. - Creators of MediaLandscape Software  
(http://www.media-landscape.com/) and partners in the revolutionary  
Croquet project (http://www.opencroquet.org/)