|
From: Nicholas N. <n.n...@gm...> - 2023-04-03 10:04:56
|
On Sun, 2 Apr 2023 at 08:55, Mark Wielaard <ma...@kl...> wrote: > > So for the original buildbot CI I set it up to only run the auxtests > (that is build gsl and run the testsuite under valgrind). This used to > work on all setups, but after a gcc (or glibc?) update some setups > started failing as you can see at: > https://builder.sourceware.org/buildbot/#/builders?tags=valgrind > > For the try builders I took out the now failing builders > (debian-testing-x86_64, fedora-arm64, fedora-x86_64, ibm-power10, > opensusetw-x86_64 and rawhide-x86_64). > https://builder.sourceware.org/buildbot/#/builders?tags=valgrind-try > > Hopefully we can figure out why the auxtests fail now on some of these > setups. > What are the auxtests? I don't remember hearing about them and a Google search didn't turn up anything helpful. I looked at some of the failing builders, they all fail like this: > for gsl_test in block cblas cdf cheb combination complex const deriv dht diff eigen err fft fit histogram ieee-utils integration interpolation linalg matrix min monte multifit multimin multiroots ntuple ode-initval permutation poly qrng rng roots sort specfunc statistics sum sys vector wavelet; do echo $gsl_test; done \ > | cmp - /home/builder/valgrind-auxtests/gsl-build-g-O3/valgrind-gsl.out || \ > diff -u /home/builder/shared/bb1-2/worker/valgrind-debian-testing-x86_64/build/auxprogs/gsl-1.6.out.x86.exp \ > /home/builder/valgrind-auxtests/gsl-build-g-O3/valgrind-gsl.out > - /home/builder/valgrind-auxtests/gsl-build-g-O3/valgrind-gsl.out differ: char 226, line 32 > --- /home/builder/shared/bb1-2/worker/valgrind-debian-testing-x86_64/build/auxprogs/gsl-1.6.out.x86.exp 2022-08-09 19:45:48.726405142 +0000 > +++ /home/builder/valgrind-auxtests/gsl-build-g-O3/valgrind-gsl.out 2023-04-02 13:43:46.609646296 +0000 > @@ -15,7 +15,6 @@ > histogram > ieee-utils > integration > -FAIL: qawo(f456) elist (7.25063790881233303e-15 observed vs 7.25922435194575979e-15 expected) > interpolation > linalg > matrix > @@ -30,6 +29,10 @@ > poly > qrng > rng > +FAIL: random32-bsd, 10000 steps (852261210 observed vs 1663114331 expected) > +FAIL: random64-bsd, 10000 steps (210970120 observed vs 864469165 expected) > +FAIL: random32-libc5, 10000 steps (367802360 observed vs 1967452027 expected) > +FAIL: random64-libc5, 10000 steps (221021662 observed vs 2106639801 expected) > roots > sort > specfunc > make[1]: Leaving directory '/home/builder/shared/bb1-2/worker/valgrind-debian-testing-x86_64/build/auxprogs' > > I don't know why it's comparing against `gsl-1.6.out.x86.exp` even on x86-64, arm64, ppc64le, and so on. So should we make a new check target that runs a subset of make > regtests? Or maybe have a "ci mode" for regtests? So you would run > CI_MODE=true make regtests and it would skip any vgtest that has > prereq: test -z "$CI_MODE" > > And then add make regtests to the ci and try buildbots? > Seems reasonable. Nick |