#5630 Share packages fail to load when several Maxima processes compile them at once

[David Scherfgen]

This has been discussed on the mailing list, but I think there's no bug report yet, so here it is.

When several Maxima processes load the same share package at the same time and the package has not been compiled yet, most of them fail to load it. This affects every package built with mk:defsystem (minpack, lapack, draw, graphs, fmin_cobyla and others) and typically hits parallel test runs on a fresh build, since maxima-local gives all runs of a build the same object directory. With SBCL 2.2.9, run from the top of a fresh build tree (paths shortened):

rm -rf binary
for i in 1 2 3 4; do
  ./maxima-local --no-init -q --batch-string='display2d:false$ load(minpack)$ minpack_solve([x^2-2],[x],[1.0]);' > out.$i 2>&1 &
done
wait
grep -E 'empty FASL|end of file|failed to load|Waiting|^\(%o3\)' out.[1-4]

out.1:MK:DEFSYSTEM: Error loading <objdir>/share/minpack/./dpmpar.fasl: attempt to load an empty FASL file:
out.1:MK:DEFSYSTEM: Error loading <objdir>/share/minpack/./enorm.fasl: attempt to load an empty FASL file:
out.1:(%o3) [[1.4142135623730947],8.881784197001252e-16,1]
out.2:MK:DEFSYSTEM: Error loading <objdir>/share/minpack/./dpmpar.fasl: attempt to load an empty FASL file:
out.2:loadfile: failed to load <build>/share/minpack/load-minpack.lisp
out.2:(%o3) minpack_solve([x^2-2],[x],[1.0])
out.3:MK:DEFSYSTEM: Error loading <objdir>/share/minpack/./dpmpar.fasl: attempt to load an empty FASL file:
out.3:loadfile: failed to load <build>/share/minpack/load-minpack.lisp
out.3:(%o3) minpack_solve([x^2-2],[x],[1.0])
out.4:MK:DEFSYSTEM: Error loading <objdir>/share/minpack/./enorm.fasl: attempt to load an empty FASL file:
out.4:loadfile: failed to load <build>/share/minpack/load-minpack.lisp
out.4:(%o3) minpack_solve([x^2-2],[x],[1.0])

Only one of the four processes returns [[1.4142135623730947],8.881784197001252e-16,1]. In the others load(minpack) fails and minpack_solve stays unevaluated. Three parallel run_testsuite(share_tests=true) runs on a fresh build reported 212, 8 and 351 failures, against 1 for a single run.

Whether a package loads must not depend on what other processes are doing. The first process to compile a file writes the binary under its final name, and the others take that empty or half-written file for an up-to-date binary and load it. The recovery for a binary that fails to load then deletes the file the first process is still writing. A process killed while compiling, for example by timeout or by heap exhaustion while compiling lapack, likewise leaves an empty or truncated binary that the next run loads.