Menu
▾
▴
Re: [LTP] [PATCH] Detect test results more accurately when generating HTML
|
From: rohit v. <roh...@gm...> - 2009-05-22 09:26:06
|
Hi,
There is no problem with the genhtml.pl script. The real problem is in the
pan driver (ltp/pan/pan.c). This seems to be a problem of race condition
after going through the code.
My suggestion was to use the buffered mode as default. This can be done with
the -O option followed by a directory name where the buffered data can be
stored. But, there is a problem - the -O option has no effect unless the -x
option with an argument greater than 1 is used.
In your test runs you can add the options " -O <dirname> -x 2" and find out
if the problem with logging still persists.
Regards,
rohit
On Fri, May 22, 2009 at 2:43 PM, Marc Gauthier <ma...@te...> wrote:
> Hi Rohit,
>
> I guess we've each observed output that the other has not :-)
>
> Per your (1): I've seen a few cases where lines between test_end and
> test_start do belong to the preceding one, though they are indeed the
> exception, not the rule. Looking at the name to assign to previous seems
> to catch the proper ones in all the cases I've seen (default test lists
> don't repeat the same test twice in a row).
>
> Per your (2): Haven't seen that, interesting. That's a bit harder to sort
> out, but the script could at least catch the lines that report
> PASS/FAIL/BROK etc. And perhaps lines that don't look like a collection of
> var=value. Hmmm.... maybe all it needs is another regexp test, see patch
> below (against the genhtml.pl file I posted, which btw was against its
> latest version, 1.3).
>
> I haven't taken time to look at why things get out of order in the first
> place, which seems like the correct thing to fix (well, I looked enough to
> notice that fflush(stdout) was really there :-). Though making the script
> handle it doesn't hurt, and I've simply been trying to get an accurate look
> at test results, this was the quickest way to do it.
>
> -Marc
>
> --- genhtml-v4.pl 2009-05-21 01:08:14.747493000 -0700
> +++ genhtml.pl 2009-05-22 02:07:47.756226000 -0700
> @@ -116,18 +116,18 @@
> }
>
> # Read test result parameters and test output
> - while ($line !~ /$end_tag/) {
> + while (1) {
> + get_line(\$line) or last TEST;
> + last if $line =~ /$end_tag/;
> ($read_output = 1, next) if $line =~ /$output_tag/;
> ($read_output = 0, next) if $line =~
> /$execution_tag/;
> - if ($read_output) {
> + if ($read_output or $line !~
> /^(\s*\w+=(".*"|\S*))+\s*$/) {
> push @output_lines, $line;
> } else {
> while ($line =~ s/^\s*(\w+)=(".*"|\S*)//) {
> $values{$1} = $2;
> }
> }
> - } continue {
> - get_line(\$line) or last TEST;
> }
>
> # Treat lines that follow <<<end_test>>> as output lines
>
>
>
>
>
> ------------------------------
> *From:* rohit verma [mailto:roh...@gm...]
> *Sent:* Friday, May 22, 2009 00:13
> *To:* su...@li...
> *Cc:* Marc Gauthier; LTP List
> *Subject:* Re: [LTP] [PATCH] Detect test results more accurately when
> generating HTML
>
> Hi,
> The suggestion to - "Process test output lines that appear between
> <<<test_end>>>
> and the following <<<test_start>>>. Assign to the preceding test where the
> name matches, else to the following test." does not solve the problem
> completely.
>
> My observation is:
>
> 1. Lines between <<<test_end>>> and <<<test_start>>> belongs to the
> following test case and not to the preceding one.
> 2. Also, at times I have observed the test output is spread in such a way
> that few lines of the test output fall between <<<test_end>>> and
> <<<test_start>>> and rest of lines appear between the <<<test_start>>> and
> <<<test_output>>> of the corresponding testcase.
>
> Regards,
> rohit
>
> On Fri, May 22, 2009 at 12:22 PM, Subrata Modak <
> su...@li...> wrote:
>
>> On Thu, 2009-05-21 at 01:13 -0700, Marc Gauthier wrote:
>> > Hi, my first patch to this list. Actually not a patch,
>> > but the files themselves, to avoid line-wrapping issues
>> > and because the diff is twice the size of the new file.
>> >
>> > -Marc
>>
>> Thanks Marc.
>>
>> Rohit,
>>
>> Can you test this and see if these solves the problem that you were
>> discussing for long ?
>>
>> Regards--
>> Subrata
>>
>> >
>> >
>> > -----------
>> > Detect test results more accurately when generating HTML
>> >
>> > Process test output lines that appear between <<<test_end>>>
>> > and the following <<<test_start>>>. Assign to the preceding
>> > test where the name matches, else to the following test.
>> >
>> > If a single test has multiple types of results (e.g. both
>> > FAIL and WARN), report only the most significant one, to
>> > avoid mis-computing the total number of PASS tests or
>> > total counts that don't add up to the number of tests.
>> >
>> > If a test's output has no explicit result (PASS, FAIL, etc),
>> > look at the exit value to determine whether it passed.
>> >
>> > Setting the SHOW_UNRESOLVED environment variable to 1
>> > classifies as UNRESOLVED any test with no explicit result
>> > and a zero exit code.
>> >
>> > Setting the SUMMARY_OUTPUT environment variable to 1
>> > causes only one line of output per test to be shown, for a
>> > tighter page that allows quickly scanning the results.
>> >
>> > Show percentage of each result type in summary section.
>> >
>> > Simplify parsing a bit.
>> >
>> > Signed-off-by: Marc Gauthier <ma...@te...>
>> >
>> >
>> ------------------------------------------------------------------------------
>> > Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
>> > is a gathering of tech-side developers & brand creativity professionals.
>> Meet
>> > the minds behind Google Creative Lab, Visual Complexity, Processing, &
>> > iPhoneDevCamp asthey present alongside digital heavyweights like
>> Barbarian
>> > Group, R/GA, & Big Spaceship. http://www.creativitycat.com
>> > _______________________________________________ Ltp-list mailing list
>> Ltp...@li...
>> https://lists.sourceforge.net/lists/listinfo/ltp-list
>>
>>
>
|