[Reduce-algebra-developers] How to build REDUCE reproducibly?

[Andrey I. <ign...@im...>]

Hello,

Almost all packages from major Linux distributions are now built 
reproducibly in the sense of 
https://reproducible-builds.org/docs/definition/. Perhaps pretty soon 
the build reproducibility may become a requirement for all the packages 
for security reasons. Here are some reasons for indeterminism I have 
found that can make REDUCE builds non-reproducible.

1. Timestamps
     a) There is a build datestamp in the banner seen every time the 
REDUCE is launched. It can be removed by performing substitution '(setq 
date!* (date))' -> '(setq date!* "")' in csl/cslbase/buildreduce.lsp.
     b) A separate build datestamp shows up when rfcsl (a REDUCE 
terminal frontend) is launched. It can be fixed by removing 'DATESTAMP = 
$(shell date +%d-%b-%Y)' in generic/newfront/Makefile.am and 
'-DBUILDTIME="\"`LANG=en date +%d-%b-%Y`\""' in generic/redfront/Makefile.in
     c) The image files csl.img, bootstrapreduce.img, and reduce.img 
containing lisp/reduce functions compiled to bytecode have a lot of 
timestamps alongside the modules names (mostly in the preamble). These 
timestamps are generated by calls to 'preserve' function in 
csl/cslbase/buildreduce.lsp. Substitution 'std::time(.*)' -> '0' in 
csl/cslbase/preserve.cpp makes all these timestamps equal to "Thu Jan  1 
00:00:00 1970", the Unix epoch, at least if your local timezone is 
UTC+00:00. Does this modification break something important?

2. According to 'redcsl -w --help' '-r NNN' flag allows to fix the 
initial seed of the internal CSL random number generator to NNN (by 
default this seed depends on "date, time of day and as much genuine 
randomness as the operating system is willing to provide"). To build 
REDUCE with fixed initial seed one can use 'make CSLFLAGS=-r NNN'. Does 
it actually help to reduce build nondeterminism?

After performing steps 1-2 REDUCE builds are still non-reproducible. All 
non-reproducibility is now contained in bootstrapreduce.img and 
reduce.img images. See the typical output of the diffoscope program (it 
is like diff) run on two independent install trees in the attachment. 
While bootstrapreduce.img has just a single nondeterministic byte, 
reduce.img has a lot of them. This is strange since according to 'Inside 
Reduce' text reduce.img should contain only part of bytcode compiled 
lisp/reduce functions present in bootstrapreduce.img (missing functions 
are translated to C and compiled to reduce executable, which is 
deterministic).

What are the reasons for this remaining nondeterminism?

Best regards,
Andrey Ignatenko