Re: [Maxima-discuss] nonzero exit on error, was: Writing tests

[Robert D. <rob...@gm...>]

On Tue, Apr 8, 2025 at 7:51 AM Leo Butler <Leo...@um...> wrote:

> > Oh, thanks, that's right, it's enabled by --quit-on-error. I wonder if
> > we should make the default. What do other systems which allow for
> > noninteractive processing do?
>
> I don't think it should be the default. For example, it would cause
> pre-mature termination of the testsuite.

Hmm, I see the effect of --quit-on-error is to terminate interactive
sessions as well -- I have to say that's pretty surprising. Doesn't it
make sense to keep going after an error in an interactive session, but
stop with a nonzero exit code in a noninteractive session? Comparable
systems that I tried (Python, R, Octave, Perl) terminate
noninteractive sessions with a nonzero exit code and continue
interactive sessions after an error. Won't users expect Maxima to work
in a similar way?

FWIW, it looks like batch(..., test) catches errors and keeps going,
even when --quit-on-error is enabled, while run_testsuite() does not
catch errors and Maxima terminates. To be honest that seems like an
oversight in run_testsuite.

> TEST-BATCH currently returns a bunch of information about the test
> outcomes, but because it is returned as a list of VALUES, only the first
> item is accessible from the Maxima code--and since that is the filename
> of the test file, the return value is useless.
>
> I could refashion $BATCH_TEST so that it returns a proper list and make
> the test information visible to Maxima code.

Yes, it makes sense to return a Maxima list comprising the data that
are already available. It is worth considering returning a list of
equations (i.e. something like [filename = "foo.mac", expected_errors
= [2, 4], actual_errors = [1]]) or a Maxima struct.

Robert