Re: [Maxima-discuss] nonzero exit on error, was: Writing tests

[Leo B. <Leo...@um...>]

On Mon, Apr 07 2025, Robert Dodier <rob...@gm...> wrote:

> On Mon, Apr 7, 2025 at 10:30 AM Leo Butler <Leo...@um...> wrote:
>
>> Yes, it is in current HEAD.
>
> Oh, thanks, that's right, it's enabled by --quit-on-error. I wonder if
> we should make the default. What do other systems which allow for
> noninteractive processing do?

I don't think it should be the default. For example, it would cause
pre-mature termination of the testsuite.

>
>> $ ./maxima-local -p src/mload.lisp --no-init --batch-string='quit(batch_test("tests/rtest1.mac"));'
>> .
>> The following 2 problems failed: (183 186)
>>
>> $ echo $?
>> 2
>
> I dunno, I'm not in favor of it. We would want to return a nonzero
> code in lots of situations, aside from test cases. I think it makes
> sense to have some visibility into the outcomes of test cases, but I
> don't think this is it.

(This is partly an answer to Ray's questions in the same thread.)

TEST-BATCH currently returns a bunch of information about the test
outcomes, but because it is returned as a list of VALUES, only the first
item is accessible from the Maxima code--and since that is the filename
of the test file, the return value is useless.

I could refashion $BATCH_TEST so that it returns a proper list and make
the test information visible to Maxima code.

If you don't like that idea, perhaps you could suggest something else
that would improve the ability to write testsuites for 3rd-party
packages.

Best regards,
Leo