Re: [LTP] [PATCH] Disable Broken Test Cases

[Subrata M. <tos...@gm...>]

Hi,

On Sun, Mar 8, 2009 at 9:59 PM, CAI Qian <ca...@cc...> wrote:

> Hi,
>
> This patch temporarily disables broken test cases,
>
> nanosleep01
> nanosleep02
> pselect01
> pselect01_64
>
> because the ways to calculate the syscall execution time looks like
> invalid. For example, in pselect01 test case,
>
>  start = time(&t);
>   retval = pselect(0, &readfds, NULL, NULL, (struct timespec *)&tv,NULL);
>  end = time(&t);
>
>  /* Changed total_sec compare to an at least vs an exact compare */
>
>  if(((end - start) >= total_sec) && ((end - start) <= total_sec + 1))
>  tst_resm(TPASS,"Sleep time was correct");
>  else
>  tst_resm(TFAIL,"Sleep time was incorrect:%d != %d",total_sec,(end -
> start));
>  }
>
> The CPU scheduler can suspend the execution of the program just after
> time(), but before pselect(). As the result, (end - start) is not the
> actual syscall execution time, which introduces a false test failure
> here. The same problem exists in nanosleep01 and nanosleep02 as well,
>
>   /* Note down the current time */
>      gettimeofday(&otime, 0);
>   /*
>    * Call nanosleep() to suspend child process
>    * for specified time.
>    */
>   TEST(nanosleep(&timereq, NULL));
>
>   /* time after child resumes execution */
>      gettimeofday(&ntime, 0);
>
>
> Signed-off-by: CAI Qian <ca...@cc...>


I am not sure whether it would be correct to disable tests when they are
broken (supposed to be). The inherent reason being that once a/some
test/tests are disabled, they just go out of the perview of the testers
memory, as, he/she can no more see the test results of the same tests.
Already overburdened to handle/analyze so many tests results, he simply
forgets that such-and-such tests - disabled some time ago due to broken
issue(s) - needs to be fixed.

So, i would prefer to keep them, so that it occasionally pricks into
somebody(s) eye(s), and he/she is generous enough to give a fix someday in
future.

Regards--
Subrata


>
> --- runtest/syscalls.orig       2009-03-09 00:04:17.000000000 +0800
> +++ runtest/syscalls    2009-03-09 00:15:49.000000000 +0800
> @@ -622,8 +622,10 @@
>  munmap02 munmap02
>  munmap03 munmap03
>
> -nanosleep01 nanosleep01
> -nanosleep02 nanosleep02
> +# The following test cases are broken by design with the syscall time
> +# calculations.
> +#nanosleep01 nanosleep01
> +#nanosleep02 nanosleep02
>  nanosleep03 nanosleep03
>  nanosleep04 nanosleep04
>
> @@ -699,8 +701,10 @@
>
>  profil01 profil01
>
> -pselect01 pselect01
> -pselect01_64 pselect01_64
> +# The following test cases are broken by design with the syscall time
> +# calculations.
> +#pselect01 pselect01
> +#pselect01_64 pselect01_64
>
>  ptrace01 ptrace01
>  ptrace02 ptrace02
>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco,
> CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the
> Enterprise
> -Strategies to boost innovation and cut costs with open source
> participation
> -Receive a $600 discount off the registration fee with the source code:
> SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Ltp-list mailing list
> Ltp...@li...
> https://lists.sourceforge.net/lists/listinfo/ltp-list
>



-- 
Regards & Thanks--
Subrata