Thread: [Module::Build] Capturing test output to please CPANPLUS

Status: Beta

Brought to you by: kwilliams

This project can now be found here.

module-build-general

[Module::Build] Capturing test output to please CPANPLUS

From: Randy W. S. <Ra...@Th...> - 2006-04-08 21:02:43

Attachments: test_capture.diff

Will something like this work for a temporary solution that we can maybe 
figure out how to extend in the future to always capture output?

It can be driven like so:

use Module::Build;

my $mb = Module::Build->new_from_context;
$mb->dispatch('build');

$ENV{PERL_MM_USE_DEFAULT} = 1; # or $ENV{PERL5_CPANPLUS_IS_RUNNING} = 1

eval{ $mb->dispatch('test') };

if ( $mb->test_captured ) {
   my $out = $mb->test_out;
   my $err = $mb->test_err;

   print <<"---";
output:
=======
$out

errors:
=======
$err
---
}

Re: [Module::Build] Capturing test output to please CPANPLUS

From: Randy W. S. <ml...@th...> - 2006-04-10 06:46:42

Randy W. Sims wrote:
> 
> Will something like this work for a temporary solution that we can maybe 
> figure out how to extend in the future to always capture output?

Regarding Ticket #9793 [1]

Scratch that. I don't really like my previous solution. I'm not sure 
Module::Build needs to be responsible for capturing output; I can't 
really see any benifit. Is there a reason CPANPLUS (or 
CPANPLUS::Dist::Build) can't do this, like below (borrowing again from 
Jos' patch), or are there other issues I'm not aware of?

Thanks,
Randy.

use strict;
use warnings;

use Module::Build;
my $mb = Module::Build->new_from_context;
$mb->dispatch('build');

my $out = '';
{
     require IO::String;
     local *STDOUT;
     my $io = tie( *STDOUT, 'IO::String' );
     eval { $mb->dispatch('test') };
     $out = $io->string_ref;
}

print $$out;
die $@ if $@;

1. <http://rt.cpan.org/Public/Bug/Display.html?id=9793>

Re: [Module::Build] Capturing test output to please CPANPLUS

From: Jos I. B. <ka...@xs...> - 2006-04-10 08:45:20

On Apr 10, 2006, at 8:46 AM, Randy W. Sims wrote:

> Randy W. Sims wrote:
>> Will something like this work for a temporary solution that we can  
>> maybe figure out how to extend in the future to always capture  
>> output?
>
> Regarding Ticket #9793 [1]
>
> Scratch that. I don't really like my previous solution. I'm not  
> sure Module::Build needs to be responsible for capturing output; I  
> can't really see any benifit.

Well as an API to a build interface, it would be nice to get the  
diagnostics back from that
API, without having to rely on retie'ing STDOUT in your client  
application -- not just CPANPLUS
will want to use this API, but every coder (else, why bother with an  
API?).

The point is that you have the data available at a very precise  
location in M::B, and all you
need to do is store it, and every user can happily use it -- make an  
accessor called ->stack()
with all your 'extra' messages? a specific one called ->test_output?;  
it's all the same to
the user, as long as it doesn't get thrown away!


> Is there a reason CPANPLUS (or CPANPLUS::Dist::Build) can't do  
> this, like below (borrowing again from Jos' patch), or are there  
> other issues I'm not aware of?

It would have to be in CPANPLUS::Dist::Build (which is under the  
module-build group care now
as well) -- It's not an elegant solution, and might break if you  
change any of the internals
of M::B (or test::harness), but it beats not having any output at  
all, since this renders test
reports sent to testers.cpan.org for authors who use M::B quite  
useless :(

> use Module::Build;
> my $mb = Module::Build->new_from_context;
> $mb->dispatch('build');
>
> my $out = '';
> {
>     require IO::String;
>     local *STDOUT;
>     my $io = tie( *STDOUT, 'IO::String' );
>     eval { $mb->dispatch('test') };
>     $out = $io->string_ref;
> }
>
> print $$out;
> die $@ if $@;
>
>
> 1. <http://rt.cpan.org/Public/Bug/Display.html?id=9793>

--
	Jos Boumans

	"Two rules to success in life:
   	1. Don't tell people everything you know"
		--Sassan Tat

	CPANPLUS	http://cpanplus.sf.net

Re: [Module::Build] Capturing test output to please CPANPLUS

From: Ken W. <ke...@ma...> - 2006-04-10 14:51:25

On Apr 10, 2006, at 1:46 AM, Randy W. Sims wrote:

> Randy W. Sims wrote:
>> Will something like this work for a temporary solution that we can  
>> maybe figure out how to extend in the future to always capture  
>> output?
>
> Regarding Ticket #9793 [1]
>
> Scratch that. I don't really like my previous solution. I'm not  
> sure Module::Build needs to be responsible for capturing output; I  
> can't really see any benifit. Is there a reason CPANPLUS (or  
> CPANPLUS::Dist::Build) can't do this, like below (borrowing again  
> from Jos' patch), or are there other issues I'm not aware of?

Probably as much as possible should go in CPP:D:Build, unless there  
are aspects of it that would also be nice to have in the core, and  
they wouldn't generate too much clutter.

In general I'm leery of messing with filehandles too much in the  
core, because the people can't do their own messing if they want to.

  -Ken

Re: [Module::Build] Capturing test output to please CPANPLUS

From: Randy W. S. <ml...@th...> - 2006-05-16 09:29:34

Attachments: cpp.diff

Ken Williams wrote:
> 
> On Apr 10, 2006, at 1:46 AM, Randy W. Sims wrote:
> 
>> Randy W. Sims wrote:
>>
>>> Will something like this work for a temporary solution that we can  
>>> maybe figure out how to extend in the future to always capture  output?
>>
>>
>> Regarding Ticket #9793 [1]
>>
>> Scratch that. I don't really like my previous solution. I'm not  sure 
>> Module::Build needs to be responsible for capturing output; I  can't 
>> really see any benifit. Is there a reason CPANPLUS (or  
>> CPANPLUS::Dist::Build) can't do this, like below (borrowing again  
>> from Jos' patch), or are there other issues I'm not aware of?
> 
> 
> Probably as much as possible should go in CPP:D:Build, unless there  are 
> aspects of it that would also be nice to have in the core, and  they 
> wouldn't generate too much clutter.
> 
> In general I'm leery of messing with filehandles too much in the  core, 
> because the people can't do their own messing if they want to.

I've been sitting on the attached patch against CPANPLUS::Dist::Build 
for a while because 1) I'm not knowledgeable about CPANPLUS to know how 
to properly test it; and 2) it still has the problem of suspending 
output until the test completes before sending to stdout.

I thought maybe I could extend the patch a little further to use IO::Tee 
to eliminate the delayed output, but it doesn't seem to work:

#!/usr/bin/perl

use File::Spec;
chdir( File::Spec->catdir( glob('~'), 'Module-Build' ) );

use Module::Build;
my $mb = Module::Build->new_from_context;
$mb->dispatch('build');


require IO::String;
my $io = IO::String->new;

require IO::Tee;
my $tee = IO::Tee->new(\*STDOUT, $io);


my $SAVE_OUT = select($tee);

$mb->dispatch('test', test_files => 't/runthrough.t', verbose => 1);

select($SAVE_OUT);


my $result = ${$io->string_ref};
# craft a report from $result . $@

__END__


This works fine when tests succeed, but when there is a failure in the 
test...

# Looks like you failed 1 test of 33.
dubious
         Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 1
         Failed 1/33 tests, 96.97% okay (less 4 skipped tests: 28 okay, 
84.85%)
Undefined format "Symbol::GEN128" called at 
/usr/share/perl/5.8/Test/Harness.pm line 542.


Anyone know how to make this work, please?

Randy.

Re: [Module::Build] Capturing test output to please CPANPLUS

From: Randy W. S. <Ra...@Th...> - 2006-05-16 22:01:09

Attachments: cpp.diff

[Resending...]

Ken Williams wrote:
> 
> On Apr 10, 2006, at 1:46 AM, Randy W. Sims wrote:
> 
>> Randy W. Sims wrote:
>>
>>> Will something like this work for a temporary solution that we can  
>>> maybe figure out how to extend in the future to always capture  output?
>>
>>
>> Regarding Ticket #9793 [1]
>>
>> Scratch that. I don't really like my previous solution. I'm not  sure 
>> Module::Build needs to be responsible for capturing output; I  can't 
>> really see any benifit. Is there a reason CPANPLUS (or  
>> CPANPLUS::Dist::Build) can't do this, like below (borrowing again  
>> from Jos' patch), or are there other issues I'm not aware of?
> 
> 
> Probably as much as possible should go in CPP:D:Build, unless there  are 
> aspects of it that would also be nice to have in the core, and  they 
> wouldn't generate too much clutter.
> 
> In general I'm leery of messing with filehandles too much in the  core, 
> because the people can't do their own messing if they want to.

I've been sitting on the attached patch against CPANPLUS::Dist::Build
for a while because 1) I'm not knowledgeable about CPANPLUS to know how
to properly test it; and 2) it still has the problem of suspending
output until the test completes before sending to stdout.

I thought maybe I could extend the patch a little further to use IO::Tee
to eliminate the delayed output, but it doesn't seem to work:

#!/usr/bin/perl

use File::Spec;
chdir( File::Spec->catdir( glob('~'), 'Module-Build' ) );

use Module::Build;
my $mb = Module::Build->new_from_context;
$mb->dispatch('build');


require IO::String;
my $io = IO::String->new;

require IO::Tee;
my $tee = IO::Tee->new(\*STDOUT, $io);


my $SAVE_OUT = select($tee);

$mb->dispatch('test', test_files => 't/runthrough.t', verbose => 1);

select($SAVE_OUT);


my $result = ${$io->string_ref};
# craft a report from $result . $@

__END__


This works fine when tests succeed, but when there is a failure in the
test...

# Looks like you failed 1 test of 33.
dubious
          Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 1
          Failed 1/33 tests, 96.97% okay (less 4 skipped tests: 28 okay,
84.85%)
Undefined format "Symbol::GEN128" called at
/usr/share/perl/5.8/Test/Harness.pm line 542.


Anyone know how to make this work, please?

Randy.