[Fwd: Re: Different types of tests (was Re: [Module::Build] [RFC] author tests)]

[David G. <da...@hy...>]

(Bringing my reply to Rob back to the list as well.)

-------- Original Message --------
Subject: Re: Different types of tests (was Re: [Module::Build] [RFC] 
author tests)
Date: Fri, 03 Feb 2006 13:39:26 -0500
From: David Golden <da...@hy...>
To: Rob Kinyon <rob...@gm...>
References: <703...@ma...> 
  <43E...@hy...> 
<703...@ma...>

It doesn't look like you copied this to the list.

A few comments in reply:

Rob Kinyon wrote:
> I don't think they're non-functional tests. They're useful tests and

I was using "functional" along the lines of "functional" and
"non-functional" requirements.  I.e. "does what it says it does".  In
that sense, making sure the Pod is correct is non-functional.  It's
confusing jargon, I'll admit.

> You're missing the point. What if I, as the maintainer of my distro,
> have a series of tests that require special setup? What about
> benchmarking tests? What about tests for a reason that only Damian can
> think of?

We may be missing each others' points.  I'm suggesting solving a
narrower problem, intentionally.

> Plus, if you decide that a given test file belongs in a different
> testing scheme, you change the Build.PL, not the test file. This means
> that the test has more integrity  because it's been touched less.

Are you really suggesting that here that a benchmarking test is going to
become a production test?

> This is a straw man and there's a very easy solution - require the
> version of MB that provides this feature. This is no different than
> the Test::Simple upgrade from 0.46 to 0.47 which had the major API
> change. Ever wonder why every Build.PL in existence has:
> 
>     build_requires => {
>         'Test::More' => '0.47',
>     },

Which as a side note is broken for many version of CPANPLUS because
CPANPLUS doesn't (didn't?) honor build_requires.  That's partly my point
about the interdependencies between the tool chain.  As another example,
how does "prove" know which tests are in which of your groups?  The
environment variable approach leaves it to tests files to know if they
should fire rather than the tools running the test files.  (There's an
interesting functional vs OO paradigm question in this, I think.)

> I do have another option - I write and maintain Module::Build::Rob
> that provides this feature. Except, I don't want to do that because:
>     a) I have enough on my plate
>     b) The name sucks and I can't think of a better one
>     c) I don't want to.

I guess the proof will be in the code.  I think it's easier to write and
maintain the environment variable approach.  I think what you're talking
about sounds great, too, but I'm not sold that the need for it justifies
the effort to create it and test it and maintain it.  If you do --
great.  If not, that also suggests a simpler approach might be better.

Regards,
David