From: Mikael S. <mik...@ax...> - 2003-01-24 21:07:27
|
As Paul points out crash0* are intentionally random to create various nasty code and calls. I guess that these tests are usually not run in a daily regression test system (not in ours anyway). But there are also other tests like fsync02 that may generate more or less BROK depending on the random values (the test should of course return 0 BROK in a correct system). Our daily regression tests with LTP returns a different number of PASS every day (but BROK and FAIL are the same). This may be caused by errors in my testscripts or by randomness in the LTP tests. Please tell me if you have the same experience. If eveybody gets the same amount of PASS/FAIL/BROK for every run I know for sure that my scripts are broken. /Mikael -----Original Message----- From: Dan Kegel To: Mikael Starvik Cc: 'ltp...@li...' Sent: 2003-01-24 22:09 Subject: Re: [LTP] Re: results comparator? (Dan Kegel) Mikael Starvik wrote: > There are some testcases that are random in their behaviour > (e.g. crash01 and crash02). IIRC there are also some tests > that runs a random number of testcases and hence generates > different number of PASS/FAIL/BROK. > > It would sure be nice if it would be possible to disable > this randomness to make the results 100% reproducable > (assuming everything outside LTP is exaclty the same). Hey, good point. Sounds like something that will be needed (if it's not already there). - Dan -- Dan Kegel http://www.kegel.com http://counter.li.org/cgi-bin/runscript/display-person.cgi?user=78045 |