Re: [Valgrind-developers] extending vg_regtest to also produce automake-flavoured result files

[Mark W. <ma...@kl...>]

Hi Frank,

On Wed, Aug 14, 2024 at 03:28:27PM -0400, Frank Ch. Eigler wrote:
> > You don't have to look at them, but they are there. So I am sure people
> > will look at them and wonder what/why they are empty. I think it would
> > be good if they weren't totally empty but contained at least some hint
> > of where to look for the "real" results. [...]
> 
> Good idea, added.

  See *.log files for details on each test in this directory.
  See /home/mark/src/valgrind/test-suite-overall.log

Nice. Thanks.

> > > Code is in the users/fche/try-vgregtest branch.
> > There are 4 commits on that branch. I assume you want to squash those?
> 
> Yes, done now.
>
> > Might be better to be precise and add test-suite.log and /test-suite-
> > overall.log separately?
> > 
> > +/test-suite-overall.log
> > +test-suite.log
> 
> Sure, done.

Thanks.
 
> > [...]
> > Kind of same question here. Why search for the test-suite-overall.log?
> > Don't we know exactly where it is? Just remove it directly?
> 
> Sure, done.
> 
> > > -sub mysystem($) 
> > > +sub mysystem($@) 
> >                    ^ extra whitespace.
> 
> OK, trimmed off all line-ending whitespace throughout file.

Thanks. Although I am not a fan of doing it for the whole file at the
same time as the rest of the patch (now it is slightly more difficult
to see what the real change is without using git show -w -b).

> > >  {
> > > -    my $exit_code = system($_[0]);
> > > +    my $tslog = shift(@_);
> > > +    print $tslog scalar localtime, " executing @_\n";
> > > +    my $exit_code = system(@_);
> > > +    print $tslog scalar localtime, " rc $exit_code\n";
> > >      ($exit_code == 2) and exit 1;      # 2 is SIGINT
> > >      return $exit_code;
> > >  } 
> > 
> > I don't know precisely of tslog works, it obviously adds a timestamp.
> > But in this case it also indents, but before instead of after the
> > timestamp, which seems odd, it results in the following:
> 
> Not quite.  What you're seeing is the indented transcription of the
> non-PASSing individual subtest .log in the test-suite-overall.log
> file.  New code prefixes all such lines with
> 
> | <tab>
>
> so it's a little more clear what's what.

Ah, nice. That makes things much clearer.
 
> > [...]
> > So all my comments are really whitespace comments :)
> 
> OK, final (?) testing underway on the users/fche/try-vgregtest branch.

I looked over the diff -w -b and as far as my perl goes, it looks
good. Also the output files look ok. And the .gitignore and make clean
parts seem to work as expected.

I haven't looked at the bunsen import, I assume you have and that it
looks fine?

One last request. Could you add a little paragraph under "Running the
regression tests" in README_DEVELOPERS about the test-suite*.log and
*.vgtest.[trs|log] files?

Thanks,

Mark