[LTP] unexpected behavior in LTP test harness

[David A. M. <dm...@re...>]

After running a test scenario file I noticed an unexpected behavior in 
the results.  After the full list of tests were complete, the first two 
tests in the list were run again.  After some investigation I found that 
two tests had not successfully built, and therefore produced an error 
message in the log:

   initiation_status="pan(406): execvp of 'modify_ldt01'
     (tag modify_ldt01) failed.  errno:2  No such file or directory"

   initiation_status="pan(406): execvp of 'modify_ldt02'
     (tag modify_ldt02) failed.  errno:2  No such file or directory"

This makes sense, since the build did not succeed so there was no binary 
to run, but there was no BROK or FAIL status in the log so it took some 
digging to find.  It also did not explain why two other (unrelated) 
tests were run twice.

After looking in pan/pan.c I found that run_child returns -1 if it fails 
to execvp a testcase.  The calling function (main) then checks the 
return status, and only decrements the number of tests to run (starts) 
if the return was _not_ -1.  After that, the testcase index (c) is 
incremented, and if greater than the actual number of tests, it is reset 
to zero:

     cpid = run_child(coll->ary[c], running + i, quiet_mode);
     if (cpid != -1) {
         ++num_active;
         if (starts > 0)
             --starts;
     }
 

     if (sequential)
         if (++c >= coll->cnt)
             c = 0;

so it all appears to be working as designed.  My question, is this the 
expected behavior, or should starts still be decremented when a testcase 
fails to run (so other tests aren't run twice)?

Note: In my case, only one instance of the tests were being run, so 
sequential was true.


Thank you,

d.marlin