reduce-algebra-developers — Discussion of development, administration and support for Reduce

Re: [Reduce-algebra-developers] Fix csl installation target and file permissions

[Arthur N. <ac...@ca...>]

Thank you - I hope that things are now as they should be! Arthur

On Tue, 19 Dec 2023, Andrey Ignatenko via Reduce-algebra-developers wrote:

> Hello all,
>
> Herewith this letter I suggest two simple SVN patches to rev. 6658.
>
> 1) install-man.patch
> Fix path to the man page file in install/uninstall targets in 
> csl/cslbase/Makefile.am.  Delete executable property for the man page file.
>
> 2) del-exec-properties.patch
> Delete unneeded executable property for a number of files in csl/reduce.doc 
> and csl/cslbase/fonts directories.
>
> Regards,
> Andrey Ignatenko
>

[Reduce-algebra-developers] Fix paths to binaries in .desktop files

[Andrey I. <and...@ma...>]

Hi,

Here is one more suggestion.  Using hardcoded paths to binaries in 
debianbuild/reduce/debian/*.desktop files is erroneous since true paths 
to binaries are determined by the configure script --prefix option. 
Just filenames of the binaries work fine here.

Regards,
Andrey Ignatenko

[Reduce-algebra-developers] Fix csl installation target and file permissions

[Andrey I. <and...@ma...>]

Hello all,

Herewith this letter I suggest two simple SVN patches to rev. 6658.

1) install-man.patch
Fix path to the man page file in install/uninstall targets in 
csl/cslbase/Makefile.am.  Delete executable property for the man page file.

2) del-exec-properties.patch
Delete unneeded executable property for a number of files in 
csl/reduce.doc and csl/cslbase/fonts directories.

Regards,
Andrey Ignatenko

[Reduce-algebra-developers] New release of Run-REDUCE: A JavaFX GUI for running REDUCE

[Francis W. <f.j...@li...>]

The latest release is version 3.1. For further information please visit https://fjwright.github.io/Run-REDUCE/. There is a guide that explains how to install and run Run-REDUCE, which lists some known issues at the end. This release includes installers for Microsoft Windows, Ubuntu and Fedora, plus a JAR file for any platform that supports JavaFX.



The main reason for this release now is that it attempts to fix a problem on version 22 (and probably also 21) of Ubuntu Linux, which fails to display the Run-REDUCE User Guide using the default web browser. To fix this (at least temporarily) I now write the HTML file to ~/Run-REDUCE_User_Guide.html instead of /tmp/UserGuide.html.


A couple of other significant updates are a facility to handle REDUCE user queries graphically via pop-up dialogues and an option to select whether menu-generated REDUCE commands appear when scrolling through previous input. (These have been in the source on GitHub for quite a long time.)



All release notes (and releases) are available at https://github.com/fjwright/Run-REDUCE/releases. As usual, feedback is welcome.



Francis

Re: [Reduce-algebra-developers] reduce on termux

[Andrey G. G. <A.G...@in...>]

On Mon, 25 Sep 2023, Andrey G. Grozin wrote:
> the bare csl reduce already has 90% of useful 
> functionality of redfront built in.
It is easy to have 100%. Istall rlwrap and write a script (say, reduce):

#!/usr/bin/env bash
exec rlwrap --always-readline $PREFIX/../home/reduce-6596/bin/redcsl

(in your case the directory may be slightly different). Move it to 
$PREFIX/bin.

Such a script cannot be redistributed together with reduce due to license 
reasons; however, each user may use it freely, not violating any license 
(rlwrap is GPL).

Andrey

[Reduce-algebra-developers] reduce on termux

[Andrey G. G. <A.G...@in...>]

Hello *,

I first installed Linux in 1994, on a 386 PC with 8 Mbytes. I used it, in 
particular, to run reduce. Now my arm64 android phone in my pocket (as 
well as my android tablet) is several orders of magnitude more powerful 
than that computer in 1994. Therefore, there is a natural desire to run 
reduce on them.

It is possible to run a full-featured Linux on an android device. Almost. 
It is termux. No rooting needed. Normally, it is a command-line Linux, 
though there is a rather complicated way to run X programs in it (in any 
case, X programs on a phone are useless; on a tablet, they may be usable). 
Don't install termux from play.google.com! Download apk from 
https://github.com/termux/termux-app/releases or 
https://f-droid.org/en/packages/com.termux/ and install it. You will have 
a command-line Linux which tries to be similar to Debian, with a rather 
large set of packages. A good screen keyboard (with ctrl, alt, tab, etc.) 
is needed; I recomment the hacker's keyboard 
https://play.google.com/store/search?q=hacker%27s%20keyboard&c=apps

Now, thanks to Arthur Norman, it is possible to build and run reduce on 
termux. Download the latest reduce from svn to your home directory in 
termux. Probably, you will want to

pkg install bash

(the default shell in termux is dash). Then run

scripts/termux-sanity-check.sh

it will install the necessory dependences. Then

./configure --with-csl --without-gui
make

Running ./configure will take some time, be patient. make will probably 
stop after some errors in libedit; nevermind, it is only needed for 
redfront, and the bare csl reduce already has 90% of useful functionality 
of redfront built in. Then

cd cslbuild/*/csl
make reduce.img

If you like Lisp, you can also make csl.image to have csl as a free 
bonus. Then return to the top-level reduce directory and say

bin/redcsl

Happy reducing! This is a native arm64 program, not some java shit.

Why would you want to have reduce in your pocket? There are many reasons. 
The most trivial one is: reduce is a far more convenient and powerful 
calculator than any android calculator. Just say

on rounded;

and input the expression you want to calculate in the familiar 
mathematical notation. Not via these horrid emulations of extinct pocket 
calculators.

Many thanks to Arthur for his great work,
Andrey

Re: [Reduce-algebra-developers] How to build REDUCE reproducibly?

[Andrey I. <ign...@im...>]

>> Thank you for provoking this and I think we HOPE things are now sorted. 

I have performed 10 additional test builds of revision 6601 on armhf and 
aarch64 architectures (5 builds each) using QEMU virtualization 
technology via 'guix build --system=architecture' command on GNU Guix. 
It took a lot of machine time since building in emulation mode may be 10 
times slower. I did not find any sign of non-reproducibility!

So, yes, it is seems like Reduce has become reproducible! I suppose the 
reproducible Reduce is much more attractive for developers and packagers 
from decent distributions and I hope it will become part of them soon. I 
am also glad that your efforts toward making Reduce reproducible 
resulted in a cleaner code.

Thank you very much for your work! That was fast!

Best regards,
Andrey Ignatenko

Re: [Reduce-algebra-developers] How to build REDUCE reproducibly?

[Andrey I. <ign...@im...>]

I have performed 5 consecutive builds of revision 6601 and they all 
produced exactly the same install tree. So far it seems to be apparently 
reproducible.

> So this part of it was "clock" in that it was variation in the time the 
> system spent getting from when it started up as far as what is in effect
>     base_clock := read_clock();
Now I see how my previous conclusion about "clock" was wrong.

Best regards,
Andrey Ignatenko

Re: [Reduce-algebra-developers] How to build REDUCE reproducibly?

[Arthur N. <ac...@ca...>]

I *THINK* that with the latest checking and building the cygwin version 
under windows 10 that the files "reduce.img" created by three consecutive 
build runs all match as well as md5sum can tell.
I have not tested on other platforms and the test builds I did were 
somewhat abbreviated - starting from where the reduce executable had been 
made. But at least I am slightly optimistic!

If anybody is really keen they can try building using different build 
paths to see how close to a match that leaves things. And try on a range 
of platforms etc. But at the very least things are now much closer to 
being "reproducible" than they were at the start of the week.

Arthur

Re: [Reduce-algebra-developers] How to build REDUCE reproducibly?

[Arthur N. <ac...@ca...>]

> SOURCE_DATE_EPOCH. ...
I now collect a date from the "$Id:" record that subversion updates on a
file version.h when that is checked in. I then check in using a script 
"scripts/commit.sh" that tends to make a frivolousx change to version.h so 
that subversion will check it in alongside all other changes.

> Thank you for mentioning the address space randomization, I didn't know about 
> that. But as I mentioned I observe exactly the same executable files reduce, 
> bootstrapreduce, and csl from build to build. Does it mean that if any 
> address randomization is present, it all happens at runtime, not buildtime?

I believe that happens as code is loaded, is well after build time and 
just before the user code is run - and it may have an effect on where 
"new" or "malloc" places data as well as where executable code is loaded 
to. The object is to make attacks that are based on knowing addresses with 
the code harder to set up. But I do not know full internal details.

> Problem with build paths nowdays is considered to be solved by simply 
> building in predictable paths. In can be just a fixed path,

That is a relief. If I am LUCKY it will turn out to be possible to avoid 
build path sensitivity, but I can now push that well down the priority 
list!

>> Well image files go through a compression process so changes in content - 
>> even if small - may lead to a different file-size. 
> That is interesting. I think that compression shutoff would be important step 
> on the path to clarification. Is it hard to switch it off?

It is a scheme that is coded in, so it is not a matter of there being a 
switch. It is hack the code to disable it. Well there will only be a few 
entrypoints it all goes through so on some sense it would be easy!

However when I was implementing the stuff that codes and decodes image 
files I had a scheme that could trace what was going in in semo-readable 
form and awakening that is a matter of "-DDEBUG_SERIALIZE=1" or some such.
Also since the executable files are identical (so far) I can use just a 
fixed version that and just run the strep that remakes .img files
several times. And if when I find divergences compare the trace files - 
which are HUGE.

==== (later) ====

I enabled serialization traces and also to make that at all feasible I set 
up to build a sort of mini-reduce image. And changed bits of the Reduce 
internals to avoid storing some other things in heap images.

I *believe* I may have sorted it. The issue is that the Reduce supervisor 
wants to support a statement "time;" that reports how much time has been 
used - and for that it reads the clock as it starts up and saves that 
value even if you do not than ask for a time measure. That starting clock 
number was ending up in the heap image. If was often a small integer and 
differing values dis not alter the image size. And SOMETIMES it would 
report the same value on consecutive startups so sometimes there could be 
agreement.  This behaviour impacts bootstrapreduce.img. I can sort it out 
by zeroing out those vars before saving an image in the build job.

I have not done proper testing yet (it is TEDIOUS) but have good hopes.
Now reduce.img will certainly be impacted by this so my changes will 
improve things there but there may be yet more to do. We are creeping 
again towards end-of-day so that is not for just now. I may just get to
where I can check in a version where bootstrapreduce.img is built cleanly 
- you mentioned a case where it had "just one" bit of variation.

So this part of it was "clock" in that it was variation in the time the 
system spent getting from when it started up as far as what is in effect
    base_clock := read_clock();

Arthur


>> And this is 
>>> presumably EITHER a case where something is still sensitive to the clock 
>>> or or
>>> sensitive to memory layout that can change with Linux memory address 
>>> randomization.
> Sometimes I have observed that two separate builds performed on different 
> days are exactly the same, including all the images. I think this allows to 
> exclude the "clock" possibility.
>
> Best regards,
> Andrey Ignatenko
>

Re: [Reduce-algebra-developers] How to build REDUCE reproducibly?

[Andrey I. <ign...@im...>]

> Well image files go through a compression process so changes in content - even if small - may lead to a different file-size.

I don't see any differences in file sizes of images from build to build.

Best regards,
Andrey Ignatenko

Re: [Reduce-algebra-developers] How to build REDUCE reproducibly?

[Andrey I. <ign...@im...>]

Thank you for your replies.

> I will regret loss of 
>> date info and will maybe look at changes that save a datestamp in the 
>> repository when the revision is checked in so that the date displayed 
>> can be the date of the latest checkin and keyed to the revision number 
>> rather than reflecting when the version was built. I can have a go at 
>> retrieving this from the "$Id:" stuff in version.h so it will in fact 
>> reflect the last checking that updated version.. The script 
>> scripts/commit.sh does that so people who use it are looked after!

For the cases when it is needed to retain timestamps the 
reproducible-builds project developed a specification 
https://reproducible-builds.org/specs/source-date-epoch/ which 
introduces SOURCE_DATE_EPOCH environment variable. SOURCE_DATE_EPOCH is 
supposed to be used in the source code in place of calls to the 'current 
time' functions. There are examples of its use in Makefile and C/C++ 
code in https://reproducible-builds.org/docs/source-date-epoch/. The 
value of SOURCE_DATE_EPOCH should be set to the last modification time 
of the source, incorporating any packaging-specific modifications, so it 
is supposed to be consumed from packaging systems. It is also possible 
to use other timestamps provided all of them are before the value of 
SOURCE_DATE_EPOCH. As they claim, the major distributions and some tools 
respect this specification.

>> BUILDING Reduce should not care about randomness, but I incline fairly 
>> strongly to a view that when run the random() function should (by 
>> default) behave differently each time. The user is given an option to 
>> fix the seed at startup if they need that. For chasing a particular bug 
>> repeatable behaviour can be vital - for proper testing and performance 
>> measurements if things that pretend to be random are not then bugs 
>> remain undiscovered and performance engineering can be based on the very 
>> particular circunstances of one non-random sequence. If the build 
>> scripts all specify "-r 1" that will make the BUILDS deterministic and 
>> anybody who needs deterministic tests can use the same flag. Will that 
>> suffice?

If build processes does not bring calls to random() this flag is 
unneeded in build scripts. But it may be kept (introduced?) there 
temporarily until we make Reduce reproducible.
> Then address space 
> randominzation during a build would very obviously be able to alter it. 
> If these days the ordering of addresses allocated by the underlying OS 
> during build are unstable then when I sort on addresses while building 
> various symbol tables etc who knows what may happen. reduce.img is 
> (these days) a serialized version of the heap. In serialize.cpp you will 
> see that references to compiled functions are processed using a CRC so I 
> can mention them using an integer handle. If addresses of all functions 
> in the executable are the same from build to build that will be 
> repeatable, but if not it will be a challange. Certainly release and 
> debug builds will put functions at different locations. Will address 
> space randomization and other "helpful" modern features intrude? Your 
> help may be useful!!!
Thank you for mentioning the address space randomization, I didn't know 
about that. But as I mentioned I observe exactly the same executable 
files reduce, bootstrapreduce, and csl from build to build. Does it mean 
that if any address randomization is present, it all happens at runtime, 
not buildtime?

>> If one builds two copies of anything in different directories then there 
>> are loads of divergencies. I do not know if people keen on 
>> reproducability want it to extend that far, so until that is explained 
>> (and other things sorted!) I will not address that issue. Certainly at 
>> present information about source paths and build locations tends to get 
Problem with build paths nowdays is considered to be solved by simply 
building in predictable paths. In can be just a fixed path, or, a path 
that contains hash of all inputs used to build a package as in GNU Guix 
and Nix OS.

> Well image files go through a compression process so changes in content 
> - even if small - may lead to a different file-size. 
That is interesting. I think that compression shutoff would be important 
step on the path to clarification. Is it hard to switch it off?

> And this is 
>> presumably EITHER a case where something is still sensitive to the clock 
>> or or
>> sensitive to memory layout that can change with Linux memory address 
>> randomization.
Sometimes I have observed that two separate builds performed on 
different days are exactly the same, including all the images. I think 
this allows to exclude the "clock" possibility.

Best regards,
Andrey Ignatenko

Re: [Reduce-algebra-developers] How to build REDUCE reproducibly?

[Arthur N. <ac...@ca...>]

The state of play today is (I believe) as follows:

If one builds two copies of anything in different directories then there 
are loads of divergencies. I do not know if people keen on reproducability 
want it to extend that far, so until that is explained (and other things 
sorted!) I will not address that issue. Certainly at present information 
about source paths and build locations tends to get captured...

If I clean up a source directory in a way I expected to be comprehensive 
and then configure and build - then copy all that somewhere else and 
repeat the process - I think I find (on Linux at least) that the csl, 
bootstrapreduce and reduce executables and csl.img end up bit for bit in 
agreement but bootstrapreduce.img and reduce.img do not even end up the 
same size from the two consecutive builds.
Well image files go through a compression process so changes in content - 
even if small - may lead to a different file-size. And this is presumably 
EITHER a case where something is still sensitive to the clock or or
sensitive to memory layout that can change with Linux memory address 
randomization.

The debugging paths for this that I can see right now look somewhat time 
consuming! I can make a trick version that does not use compression on 
image files so that narrowing down what changes is easier. I can enable 
the trace of serialization that will generate maybe gigabytes of log as 
images files are created. I can make trick "mini reduce" versions building 
with fewer and fewer packages in the hope tha that ends with a smaller 
example to look into. Or eyeballing code a lot.

Unless inspiration hits me or somebody responds to this with a comment 
that helps enough I will go and mull over this and work on othet projects 
for a bit... and hope to remember or be prodded into coming back to it 
later.

Of course as almost always when the cause of the variation is uncovered it 
will probably be obvious!

Arthur

Re: [Reduce-algebra-developers] How to build REDUCE reproducibly?

[Arthur N. <ac...@ca...>]

The undepinning reason for most of this was that I wanted best possible 
tracking to an exact built version when any error might get reported, so I
stuck in checksums and date-stamps in lots of places lest there be a 
divergence between the version an issue was reported on an mine where I 
was testing. You are explaining to me that the world has changed - and I 
should change with it. So reproducability also demands an unchanging 
version of all compilers and statically-linked libraries involved.

Thank you for your work identifying places that at present add 
date-stamps etc. I will interleave a few comments with your report and 
while I many not get everything sorted instantly I can try to move in the 
suggested direction!

On Tue, 19 Sep 2023, Andrey Ignatenko wrote:

> Hello,
>
> Almost all packages from major Linux distributions are now built reproducibly 
> in the sense of 
> https://reproducible-builds.org/docs/definition/. 
> Perhaps pretty soon the build reproducibility may become a requirement for 
> all the packages for security reasons. Here are some reasons for 
> indeterminism I have found that can make REDUCE builds non-reproducible.
>
> 1. Timestamps
>    a) There is a build datestamp in the banner seen every time the REDUCE is 
> launched. It can be removed by performing substitution '(setq date!* (date))' 
> -> '(setq date!* "")' in csl/cslbase/buildreduce.lsp.

So the "subversion revision number" will remain. I will regret loss of 
date info and will maybe look at changes that save a datestamp in the 
repository when the revision is checked in so that the date displayed can 
be the date of the latest checkin and keyed to the revision number rather 
than reflecting when the version was built. I can have a go at retrieving 
this from the "$Id:" stuff in version.h so it will in fact reflect the 
last checking that updated version.. The script scripts/commit.sh does 
that so people who use it are looked after!

>    b) A separate build datestamp shows up when rfcsl (a REDUCE terminal 
> frontend) is launched. It can be fixed by removing 'DATESTAMP = $(shell date 
> +%d-%b-%Y)' in generic/newfront/Makefile.am and '-DBUILDTIME="\"`LANG=en date 
> +%d-%b-%Y`\""' in generic/redfront/Makefile.in

redfront is rather separate but I may again like the though of extracting 
info from "$Id:".

>    c) The image files csl.img, bootstrapreduce.img, and reduce.img 
> containing lisp/reduce functions compiled to bytecode have a lot of 
> timestamps alongside the modules names (mostly in the preamble). These 
> timestamps are generated by calls to 'preserve' function in 
> csl/cslbase/buildreduce.lsp. Substitution 'std::time(.*)' -> '0' in 
> csl/cslbase/preserve.cpp makes all these timestamps equal to "Thu Jan  1 
> 00:00:00 1970", the Unix epoch, at least if your local timezone is UTC+00:00. 
> Does this modification break something important?

One intended purpose for those was so that there could be in efffect a 
within-reduce "make"-like scheme for updating modules, so those are the 
datestamps that one would compare source file-dates against to see what 
might need re-building. I at least hardly ever rebuild just one module and 
I have not set myself up with scripts to support it. If those had to be 
set to epoch zero then the fields should just be removed. For repeatable 
builds it would make more sense for all of them to be forced to the 
checkin date as mentioned in your point (a). I will think about a build 
option that does that (supposing you believe it would suffice).

> 2. According to 'redcsl -w --help' '-r NNN' flag allows to fix the initial 
> seed of the internal CSL random number generator to NNN (by default this seed 
> depends on "date, time of day and as much genuine randomness as the operating 
> system is willing to provide"). To build REDUCE with fixed initial seed one 
> can use 'make CSLFLAGS=-r NNN'. Does it actually help to reduce build 
> nondeterminism?
>
BUILDING Reduce should not care about randomness, but I incline fairly 
strongly to a view that when run the random() function should (by default) 
behave differently each time. The user is given an option to fix the seed 
at startup if they need that. For chasing a particular bug repeatable 
behaviour can be vital - for proper testing and performance measurements 
if things that pretend to be random are not then bugs remain undiscovered 
and performance engineering can be based on the very particular 
circunstances of one non-random sequence. If the build scripts all specify 
"-r 1" that will make the BUILDS deterministic and anybody who needs 
deterministic tests can use the same flag. Will that suffice?

> After performing steps 1-2 REDUCE builds are still non-reproducible. All 
> non-reproducibility is now contained in bootstrapreduce.img and reduce.img 
> images. See the typical output of the diffoscope program (it is like diff) 
> run on two independent install trees in the attachment. While 
> bootstrapreduce.img has just a single nondeterministic byte, reduce.img has a 
> lot of them. This is strange since according to 'Inside Reduce' text 
> reduce.img should contain only part of bytcode compiled lisp/reduce functions 
> present in bootstrapreduce.img (missing functions are translated to C and 
> compiled to reduce executable, which is deterministic).
>
> What are the reasons for this remaining nondeterminism?
>
At present I do not know! It is not something I have had cause to think 
about before. But ha ha in the past there was a stage when reduce.img was 
a dump of the memory region that everything lived in together with just 
enough info to make it possible to relocate it (or indeed to convert it 
between 32 and 64-bit needs!). Then address space randominzation during a 
build would very obviously be able to alter it. If these days the ordering 
of addresses allocated by the underlying OS during build are unstable then 
when I sort on addresses while building various symbol tables etc who 
knows what may happen. reduce.img is (these days) a serialized version of 
the heap. In serialize.cpp you will see that references to compiled 
functions are processed using a CRC so I can mention them using an integer 
handle. If addresses of all functions in the executable are the same from 
build to build that will be repeatable, but if not it will be a challange. 
Certainly release and debug builds will put functions at different 
locations. Will address space randomization and other "helpful" modern 
features intrude? Your help may be useful!!!

> Best regards,
> Andrey Ignatenko
>
I will look at the easier issues first...

Arthur

[Reduce-algebra-developers] How to build REDUCE reproducibly?

[Andrey I. <ign...@im...>]

Hello,

Almost all packages from major Linux distributions are now built 
reproducibly in the sense of 
https://reproducible-builds.org/docs/definition/. Perhaps pretty soon 
the build reproducibility may become a requirement for all the packages 
for security reasons. Here are some reasons for indeterminism I have 
found that can make REDUCE builds non-reproducible.

1. Timestamps
     a) There is a build datestamp in the banner seen every time the 
REDUCE is launched. It can be removed by performing substitution '(setq 
date!* (date))' -> '(setq date!* "")' in csl/cslbase/buildreduce.lsp.
     b) A separate build datestamp shows up when rfcsl (a REDUCE 
terminal frontend) is launched. It can be fixed by removing 'DATESTAMP = 
$(shell date +%d-%b-%Y)' in generic/newfront/Makefile.am and 
'-DBUILDTIME="\"`LANG=en date +%d-%b-%Y`\""' in generic/redfront/Makefile.in
     c) The image files csl.img, bootstrapreduce.img, and reduce.img 
containing lisp/reduce functions compiled to bytecode have a lot of 
timestamps alongside the modules names (mostly in the preamble). These 
timestamps are generated by calls to 'preserve' function in 
csl/cslbase/buildreduce.lsp. Substitution 'std::time(.*)' -> '0' in 
csl/cslbase/preserve.cpp makes all these timestamps equal to "Thu Jan  1 
00:00:00 1970", the Unix epoch, at least if your local timezone is 
UTC+00:00. Does this modification break something important?

2. According to 'redcsl -w --help' '-r NNN' flag allows to fix the 
initial seed of the internal CSL random number generator to NNN (by 
default this seed depends on "date, time of day and as much genuine 
randomness as the operating system is willing to provide"). To build 
REDUCE with fixed initial seed one can use 'make CSLFLAGS=-r NNN'. Does 
it actually help to reduce build nondeterminism?

After performing steps 1-2 REDUCE builds are still non-reproducible. All 
non-reproducibility is now contained in bootstrapreduce.img and 
reduce.img images. See the typical output of the diffoscope program (it 
is like diff) run on two independent install trees in the attachment. 
While bootstrapreduce.img has just a single nondeterministic byte, 
reduce.img has a lot of them. This is strange since according to 'Inside 
Reduce' text reduce.img should contain only part of bytcode compiled 
lisp/reduce functions present in bootstrapreduce.img (missing functions 
are translated to C and compiled to reduce executable, which is 
deterministic).

What are the reasons for this remaining nondeterminism?

Best regards,
Andrey Ignatenko

Re: [Reduce-algebra-developers] Trying to build reduce on termux

[Arthur N. <ac...@ca...>]

I will worry that the android devices I have do not have too much space. 
With termux can I access the environment eg over ssh from the keyboard and 
screen I will be comfortable with? If using it involves using an on-screen 
keyboard my enthusiasm will probebly be low enough that I will leave most 
of the debugging to you! Arthur


On Tue, 12 Sep 2023, Andrey G. Grozin wrote:

> Hello *,
>
> If you haven't heard it before, termux is a linux-like system running under 
> android. It has usual development tools (bash, llvm/clang, make, autoconf, 
> libtool and so on).
>
> I've downloaded the current reduce snapshot (r6590) to termux on my android 
> tablet and run
>
> ./configure --with-csl --without gui
>
> It seems to run normally. Then I run
>
> make
>
> and got the output
>
> /data/data/com.termux/files/bin/sh scripts/make.sh
> MFLAGS=<> MKFLAGS=<> MAKECMDGOALS=<> args=<>
> Current machine tag is aarch64-unknown-linux-gnu
> Will build for cslbuild/*aarch64-unknown-linux-gnu*/csl
>
> Then, immediately,
>
> Reduce build tasks finished.
>
> Nothing has been built, in fact. Any ideas how to proceed?
>
> It would be interesting to have reduce as a termux packege which can be 
> installed by anybody by "pkg install reduce". This could increase the number 
> of reduce users.
>
> Andrey
>
>
> _______________________________________________
> Reduce-algebra-developers mailing list
> Red...@li...
> https://lists.sourceforge.net/lists/listinfo/reduce-algebra-developers
>

[Reduce-algebra-developers] Trying to build reduce on termux

[Andrey G. G. <A.G...@in...>]

Hello *,

If you haven't heard it before, termux is a linux-like system running 
under android. It has usual development tools (bash, llvm/clang, make, 
autoconf, libtool and so on).

I've downloaded the current reduce snapshot (r6590) to termux on my 
android tablet and run

./configure --with-csl --without gui

It seems to run normally. Then I run

make

and got the output

/data/data/com.termux/files/bin/sh scripts/make.sh
MFLAGS=<> MKFLAGS=<> MAKECMDGOALS=<> args=<>
Current machine tag is aarch64-unknown-linux-gnu
Will build for cslbuild/*aarch64-unknown-linux-gnu*/csl

Then, immediately,

Reduce build tasks finished.

Nothing has been built, in fact. Any ideas how to proceed?

It would be interesting to have reduce as a termux packege which can be 
installed by anybody by "pkg install reduce". This could increase the 
number of reduce users.

Andrey

Re: [Reduce-algebra-developers] Where to find documentation about REDUCE command line arguments?

[Andrey G. G. <A.G...@in...>]

I usually use rfpsl. It runs psl reduce but provides command editing and 
history - very convenient for interactive work (there is also rfcsl).

My typical workflow is: I have 2 windows, emacs and terminal, open. I edit 
my current reduce program in emacs (many thanks to Francis Wright for a 
nice emacs mode!), save it and switch to the terminal. There I do 
<up-arrow> <enter> (this recalls the command rfpsl and starts it), then 
again <up-arrow> <enter> (this recalls the reduce command
in "foo.red";
and starts it). Then I examine results interactively in reduce, see that 
something haven't worked in the expected way, switch to emacs and correct 
my program. And so on.

Andrey

Re: [Reduce-algebra-developers] Where to find documentation about REDUCE command line arguments?

[Arthur N. <ac...@ca...>]

I probably say "ha ha" as a hollow laugh. So this email is a bit of 
explanation and things may not be as convenient as would be ideal!

Reduce is an algebra engine and at a sufficiently low level it is built on 
a kernel that acts as a Lisp system. The algebra code can be built on top 
of a range of different "lisps". Probably the most visible ones are "psl" 
and "csl", and redpsl and redcsl invoke the Reduce built atop each. Both 
are provided because around the edges those two Lisps have enough 
differences that some people may prefer one and some the other. Here is 
not the place to get entangled in fine details.
But also there is the web-reduce which us build via javascript/wasm etc 
and is separate, there has been a Reduce that hosts on to of emacs using 
the Lisp embedded within emacs, it is possible to host Reduce on a Common 
Lisp and there are or have been a number of other options. For the 
projects as a whole this offers a degree of resilience in case support for 
one of the options fails! And eg web-reduce may be slightly slower but the 
way of accessing it may be especially valuable to some... etc etc etc.
The command-line arguments belong at the level of the lisp kernel rather 
than as part of Reduce itself.

For PSL that means that in the full Reduce tree fetched perhaps using 
subversion or as a tar file of all sources one can find
    .../psl/pc-oper.pdf or .../psl/unix-oper.pdf
in the subdirectory of the full system sources that is to do with PSL and 
hence redpsl.

For CSL there are two options. The first is that if one invokes the CSL 
variant of Reduce at a suitably low level with "--help" it reports 
options. And the source file .../csl/cslbase/csl.cpp contains all the text
used there.

HOWEVER when I checked that just now if you go "redcsl --help" the help 
text appears in a window that instantly vanishes. It was supposed to have 
a pause to give people a chance to read it but it looks as if somehow that 
had got lost. I may look into that. HOWEVER (again) for redcsl the "-w" 
option asks the system to run as a console-mode app so "redcsl -w --help" 
(from a terminal) is more helpful.

For redpsl I tend to go
    redpsl < input.red | tee output.log
or in my input file I use 'out "outfile.log";'

For redcsl I can go
    redcsl -w input.red -L output.log
(for match mode I will generally be working from a console) where "-L" 
sends a copy of what goes to the screen to that file, or
    redcsl -w input.red -- output.file
in which case the output does not appear on the terminal - it JUST goes to 
the file.

The key command-line argument you may wish to override for redpsl is the 
one that sets how much memory it uses (-td) but in reality you only need 
worry about that if your calculations are HUGE and the default setting 
ends up failing saying it had run out of memory.

The web-reduce, emacs, common-lisp etc versions - well I leave you to 
investigate further!

I hope this lets you move forward a bit.

Arthur



On Wed, 28 Jun 2023, Martin Vahi wrote:

>
> After extensive surfing at REDUCE home page,
> "googling" and trying the
>
>    C:\Program Files\Reduce\bin\redpsl.bat
>
> with strings like
>
>    -?
>    --help
>    -help
>    -h
>
> I still struggle to figure out, where to get
> a list of command line arguments that I can
> give to REDUCE from a bat/Bash/whatever_shell
> The documentation seems to be all about interactive
> use of REDUCE, but I want to do
>
>    start_reduce ./my_reducefile.txt
>    some_program_written_by_me < calculation_results_by_my_reducefile_txt_code.txt
>
> and all of that NON-interactively. A lot like
>
>    ruby my_program.rb
>    python my_program.py
>    bash this_is_a_bash.script
>    reduce_or_some_wrapper_to_it ./my_reduce_code.txt
>
> Thank You.
>
> Yours sincerely,
> Mar...@so...
>
>
> _______________________________________________
> Reduce-algebra-developers mailing list
> Red...@li...
> https://lists.sourceforge.net/lists/listinfo/reduce-algebra-developers
>

Re: [Reduce-algebra-developers] Where to find documentation about REDUCE command line arguments?

[Martin G. <mar...@ic...>]

I don't have access to microsoft windows, but on Linux the corresponding
redpsl script calls bpsl which also does not respond to --help, -h, -? 
and just passes whatever arguments you specify to the reduce image. 
redcsl --help does work.

Since you are running a .bat file, did you try /help, /h or /?

I also prefer running in batch and use the following script, perhaps it
works for you with tweaking of paths. The script allows me to specify
either psl or csl and a list of scripts, each of which is executed in a
separate session and produces a transcript by appending "rlg" to the 
script name. If no files are specified it starts an interactive session.

Regards,
Martin

#!/bin/sh

# function to print usage note
function usage {
     echo "usage: reduce [-h] -c|p [-o 'opt1 ... optN'] [file1 ... fileN]"
     echo -e "\t-h	display this message and exit"
     echo -e "\t-c	use redcsl"
     echo -e "\t-p	use redpsl\n"
     echo -e "\t-o	options for redcsl\n"
     echo -e "\tIf files are specified, they are executed in separate\n"
     echo -e "\tREDUCE sessions and separate output files\n"
     echo -e "\t(program_name.rlg) are generated."
}

# Handle options
typeset porc=""
typeset cslopt=""
while getopts ":hcpo:" opt; do
    case $opt in
        h ) usage ;;
        c ) typeset reduce_exec="redcsl --nogui"
	   porc=CSL;;
        p ) typeset reduce_exec=redpsl
	   porc=PSL;;
        o ) cslopt="$OPTARG";
	   reduce_exec="redcsl --nogui $OPTARG" ;;
        \?) usage
	   exit 1 ;;
    esac
done
shift $((OPTIND - 1))

## if no file specified, run by default w/o gui
if [ -z $1 ] ;then
     if [ "$porc" == "CSL" ] ;then reduce_exec="redcsl --nogui $cslopt" ; fi
     /usr/local/bin/$reduce_exec
     exit $?
fi

if [ -z "$reduce_exec" ] ; then
     usage
     exit 1
fi

for reduce_pgm in $* ; do
     typeset name=${reduce_pgm%*.red}
     /usr/local/bin/$reduce_exec < $name.red > $name.rlg 2>&1
done



On 6/28/23 11:48, Martin Vahi wrote:
> 
> After extensive surfing at REDUCE home page,
> "googling" and trying the
> 
>      C:\Program Files\Reduce\bin\redpsl.bat
> 
> with strings like
> 
>      -?
>      --help
>      -help
>      -h
> 
> I still struggle to figure out, where to get
> a list of command line arguments that I can
> give to REDUCE from a bat/Bash/whatever_shell
> The documentation seems to be all about interactive
> use of REDUCE, but I want to do
> 
>      start_reduce ./my_reducefile.txt
>      some_program_written_by_me < calculation_results_by_my_reducefile_txt_code.txt
> 
> and all of that NON-interactively. A lot like
> 
>      ruby my_program.rb
>      python my_program.py
>      bash this_is_a_bash.script
>      reduce_or_some_wrapper_to_it ./my_reduce_code.txt
> 
> Thank You.
> 
> Yours sincerely,
> Mar...@so...
> 
> 
> _______________________________________________
> Reduce-algebra-developers mailing list
> Red...@li...
> https://lists.sourceforge.net/lists/listinfo/reduce-algebra-developers

[Reduce-algebra-developers] Where to find documentation about REDUCE command line arguments?

[Martin V. <mar...@so...>]

After extensive surfing at REDUCE home page,
"googling" and trying the

    C:\Program Files\Reduce\bin\redpsl.bat

with strings like

    -?
    --help
    -help
    -h

I still struggle to figure out, where to get
a list of command line arguments that I can
give to REDUCE from a bat/Bash/whatever_shell
The documentation seems to be all about interactive
use of REDUCE, but I want to do

    start_reduce ./my_reducefile.txt
    some_program_written_by_me < calculation_results_by_my_reducefile_txt_code.txt

and all of that NON-interactively. A lot like

    ruby my_program.rb
    python my_program.py
    bash this_is_a_bash.script
    reduce_or_some_wrapper_to_it ./my_reduce_code.txt

Thank You.

Yours sincerely,
Mar...@so...

Re: [Reduce-algebra-developers] Remove bundled libraries from REDUCE source code

[Arthur N. <ac...@ca...>]

All your questions are sensible, but the answers are more delicate.
In generate the source code for libraries is there for reasons where 
which is more important may differ case by case:
(1) I tend to be maybe extra cautious about license terms, so in some 
cases the sources of LGPL code is included so one can be really certain 
that people can build - for instance historically not everything is 
automatically present with an OS standard distribution. So putting the 
source code there felt "safe" as well as giving full credit to its author.
(2) It may be that now "every worthwhile" environment comes with a package 
manager that provides all those. Across the lifetime of Reduce that has 
not been the case. And so for any ONE particular OS some may be irrelevant 
if one wants maximum portability including to fairly archic or odd targets 
merely deleting a package would hurt. With limited available bandwidth I 
do not often look back to see what can now be purged... maybe some could!
(3) I wanted to customise FOX but the license terms mean that I have to 
adhere to LGPL and so the source of my adjusted version has to be 
available. For editline I am not aware of a single version that works 
happily on both Linux and Windows so there are 2 versions - for that and 
several other packages the build scripts I wanted were not as per the ones 
provided. Eg for libffi I want to make "universal" code for use of the 
arm-based macs and that involved trickery. Crlibm is not somewhat orphaned 
but the later work of the people who built it did not seem to be usable 
for my purposes. I know that at present I use an out of date softfloat but 
when I looked to bump the version that was going to take me more time than 
I had when I looked at it.

So if you have a system-provided libffi and editline or crlibm and 
probably softfloat they may work. My adjustments to FOX are more dramatic.
There are not liable to be performance issues.

It would be imaginable for each to have a configure-time option such as
   --with-system-libffi
possibly even making that default to true if a system version was 
available. Please join in and propose changes to configure.ac and 
Makefile.am to activate that idea! For me it would look like and extra 
number of hours in my life used up and (worse) a fresh set of potential 
variations in how things got built to need to support when future library 
updates introduce incompatibility!

So when you just look at Guix I agree that using the Guix-provided 
versions where you can would be great. I hope you can see how I am 
personally less enthusiastic about working towards it! But as I say, if 
you can hack configure.ac/Makefile.am in a nice way to provide options to 
use system versions I would be rather happy and could review what you end 
up with and check it in...

Arthur

On Thu, 1 Jun 2023, Andrey Ignatenko wrote:

> Hello,
>
> I am building REDUCE from source code in GNU Guix operation system/package 
> manager. Currently I have written a package definition which builds CSL 
> version of REDUCE with FOX GUI successfully, at least on my laptop with Intel 
> x86_64 architecture. Now I have realized that REDUCE source code contains a 
> few bundled libraries, such as fox, editline, crlibm, softfloat, etc. Since 
> GNU Guix packaging guidelines recommend to package such libraries separately, 
> here I have the following questions.
>
> 1. As far as I understand some of these libraries are customized/modified 
> versions of the original ones. Is it possible to compile REDUCE with original 
> versions of these libraries as external dependencies without impact on 
> performance?
>
> 2. Which of these bundled libraries can be easily replaced with their 
> external counterparts?
>
> 3. What modifications to the build scripts should be made to compile with 
> external libraries?
>
> Thank you.
>
>
> Best Regards,
> Andrey Ignatenko
>
>
> _______________________________________________
> Reduce-algebra-developers mailing list
> Red...@li...
> https://lists.sourceforge.net/lists/listinfo/reduce-algebra-developers
>

[Reduce-algebra-developers] Remove bundled libraries from REDUCE source code

[Andrey I. <ign...@im...>]

Hello,

I am building REDUCE from source code in GNU Guix operation 
system/package manager. Currently I have written a package definition 
which builds CSL version of REDUCE with FOX GUI successfully, at least 
on my laptop with Intel x86_64 architecture. Now I have realized that 
REDUCE source code contains a few bundled libraries, such as fox, 
editline, crlibm, softfloat, etc. Since GNU Guix packaging guidelines 
recommend to package such libraries separately, here I have the 
following questions.

1. As far as I understand some of these libraries are 
customized/modified versions of the original ones. Is it possible to 
compile REDUCE with original versions of these libraries as external 
dependencies without impact on performance?

2. Which of these bundled libraries can be easily replaced with their 
external counterparts?

3. What modifications to the build scripts should be made to compile 
with external libraries?

Thank you.


Best Regards,
Andrey Ignatenko

Re: [Reduce-algebra-developers] [reduce-algebra:discussion] Re: Reduce-Snapshot_6547.dmg does not work on macOS 10.15.7

[Jeff J. <tr...@po...>]

A quick update - I was able to build Clang 15.0.7 for macOS 10.13 - although I’m not sure it was worth the hassle.

I’ve uploaded a new REDUCE build produced with it, replacing the previous one:   https://github.com/reduce-algebra/homebrew-reduce-algebra/releases/download/Reduce_6547-x86_64-mac_10.13_highsierra-darwin17.7.0/Reduce_6547-x86_64-mac_10.13_highsierra-darwin17.7.0.dmg

This build also corrects the minor issue of libiconv being inadvertently dynamically linked so (besides the necessary linking to libc++ and libSystem) it’s fully static as intended.

Re: [Reduce-algebra-developers] [reduce-algebra:discussion] Reduce-Snapshot_6547.dmg does not work on macOS 10.15.7

[Jeff J. <tr...@po...>]

I’ve also managed to produce a new (unofficial) legacy build (this one actually done on a 10.13 system) …

It’s available at  https://github.com/reduce-algebra/homebrew-reduce-algebra/releases/download/Reduce_6547-x86_64-mac_10.13_highsierra-darwin17.7.0/Reduce_6547-x86_64-mac_10.13_highsierra-darwin17.7.0.dmg

The bad news: getting my Homebrew-based localbrew setup working on High Sierra was not for the faint of heart (and will only get degrade over time as there are no more updates for the operating system or toolchains.) Increasing the optimization level or enabling LTO, even with the latest Xcode available for this system, which is 10.1, results in a linker crash. As a result, the build targets all Intel processors generically, and uses standard optimizations.

The good news: it was done via localbrew, so the build environment is transportable.  If anyone wants to reproduce this for themselves on their own system, I can provide the whole build environment as a (rather large) archive.

Also, it might be possible to build newer versions of LLVM and Clang for better results, but really, these systems should be upgraded (or retired) soon.

I’ll add any new builds to https://github.com/reduce-algebra/homebrew-reduce-algebra/wiki/Package-Building

--
Jeffrey H. Johnson
tr...@po...

On Tue, Mar 21, 2023, at 3:08 PM, Tilda A. Steiner wrote:
> Dear Eberhard,
> 
> thank you very much for your work. I installed the legacy-snapshot_6550.dmg on macOS 10.15.7, and it seems to be working fine: both all the csl apps and psl when called from within a TeXmacs session. I guess it will probably also run on macOS 10.13.6. I will let you know if it doesn't.
> 
> 
> Cheers
> Tilda
>