I'm writing some functions for logic and I found a serious bug in simplification of logical operators:
(%i1) f(p) := not p;
(%o1) f(p):=not p
(%i2) f(not q);
(%o2) q
(%i3) f(not p);
Maxima encountered a Lisp error:
Stack overflow on value stack.
Automatically continuing.
To enable the Lisp debugger set *debugger-hook* to nil.
The problem is that when not p is simplified, p is evaluated to not p and then simplified. But here p is evaluated to not p again and simplified, so we get an infinite recursion. I think that the code which simplifies logical operators is evaluating stuff to often.
I looked at the code for evaluating predicates (src/compar.lisp) and I can't see any simple way to fix this problem.
It could be fixed either by reworking the predicates code (e.g. making Boolean operators simplifying instead of evaluating), or by changing Maxima's default scope rule from dynamic to lexical. Both of those are plenty of work. Probably the slightly easier one is to rework the predicate code.