clisp-devel — closed list for developers and testers

Cygwin build fix

[Vibhu M. <vi...@ho...>]

Hi,

I've found and fixed a problem in lispbibl.d. I've attached an 
indicative diff of the C file.

System: Windows 10, a fresh Cygwin, and the latest GitLab clisp.

I followed the instructions in INSTALL.windows ("3. Binaries for the 
Cygwin environment"), for 64-bit binaries. Besides the prescribed 
configure options, I added --ignore-absence-of-libsigsegv.

Make fails to produce spvw.o because lispbibl.c does this near line 1900:
#define CR 13
and then #includes a file (unix.c) that ultimate #includes
/usr/include/w32api/winnt.h
which itself has a legitimate symbol (not macro) named CR that gets 
replaced by 13 resulting in a syntax error.

My fix is to move the whole #define block a few lines down to after the 
#include block. That prevents the #define macros from inadvertently 
capturing symbols from the included files.

----
I'm not an expert C programmer, but this technique of #including a file 
after #defining a macro seems wrong in general. It makes changing the 
included file impossibly hard as its author isn't free to use any new 
symbols without fearing that an includer like clisp will capture them. 
If clisp files followed a rule of never allowing #includes to follow 
#defines, but to only ever precede them, I think that would consistently 
prevent this particular sort of variable capture regardless of the depth 
of an include hierarchy. An alternative prevention would be if include 
file authors only created symbols with some namespace prefix, though 
that would make their code unreadable.


With best wishes,

Vibhu

Re: Build failure on Windows 10 with current version from Git

[Robert D. <rob...@gm...>]

Wolfgang, looks like you bumped into the same error about the eof
symbol which I encountered. Here's my patch for working around that
error. That's enough to quiet that problem, although after that I ran
into other compiler errors which seemed more difficult to solve; I
don't remember what they were.

Hope this helps,
Robert

Re: Build failure on Windows 10 with current version from Git

[Wolfgang D. <wol...@da...>]

Am 17.03.22 um 18:21 schrieb Bruno Haible:
> The file INSTALL.windows (at the top-level in the git repository)
> lists the supported ways of building CLISP on Windows. MSYS is *not*
> one of the options. But mingw is, with Cygwin as a host environment.

Hi all,

I did try to setup that (build Clisp using Cygwin as host environment) 
using Github actions: https://github.com/daute/clisp-crossbuild

Yes, no optional libs (libffcall, libsigsegv, ...) are used (for now), 
that are just the first steps and I can extend the build, when I got it 
working.

(I tried a 32 bit build (--host=i686-w64-mingw32) first, as Robert did 
get problems with pointer sizes...)


The current Clisp git version gets stuck when using threads:
https://github.com/daute/clisp-crossbuild/runs/5766572996

(I think, one needs a Github account to see the log files).


When using the option  "--disable-threads" the configure step works, but 
the 'make' stage gets a problem, because 'eof' seems to be used in a 
Mingw header file:
https://github.com/daute/clisp-crossbuild/runs/5773368096
../utils/gctrigger.c:505:3: error: ‘eof’ redeclared as different kind of 
symbol
   505 |   eof,
       |   ^~~
In file included from /home/runneradmin/clisp/build/gllib/stdlib.h:59,
                  from 
/usr/i686-w64-mingw32/sys-root/mingw/include/sec_api/stdlib_s.h:9,
                  from 
/usr/i686-w64-mingw32/sys-root/mingw/include/stdlib.h:765,
                  from /home/runneradmin/clisp/build/gllib/stdlib.h:36,
                  from ../utils/gctrigger.c:36:U
/usr/i686-w64-mingw32/sys-root/mingw/include/io.h:334:15: note: previous 
declaration of ‘eof’ with type ‘int(int)’
   334 |   int __cdecl eof(int _FileHandle) 
__MINGW_ATTRIB_DEPRECATED_MSVC2005;
       |               ^~~
make: *** [Makefile:687: gctrigger.exe] Error 1


Yes, I can probably patch that (I assume, just rename the symbol 'eof' 
to some other name will be enough), but maybe that is interesting for 
the main clisp repository too...

Best regards, Wolfgang

Re: Build failure on Windows 10 with current version from Git

[Robert D. <rob...@gm...>]

On Thu, Mar 17, 2022 at 10:21 AM Bruno Haible <br...@cl...> wrote:

> The file INSTALL.windows (at the top-level in the git repository)
> lists the supported ways of building CLISP on Windows. MSYS is *not*
> one of the options. But mingw is, with Cygwin as a host environment.

Thanks for the hints. I uninstalled Msys and installed MinGW packages
via Cygwin as noted in INSTALL.windows. Now the gcc version is:

x86_64-w64-mingw32-gcc (GCC) 11.2.0

However, I get the same errors as before, namely:

intparam.h:32:2: error: #error "Type char * does not fit into a long!!"
intparam.h:33:2: error: #error "Type long * does not fit into a long!!"
intparam.h:34:2: error: #error "Type function * does not fit into a long!!"

and

spvw_mmap.d:306:11: error: conflicting types for 'mmap_prepare'; have
'int(aint *, aint *, _Bool)' {aka 'int(long unsigned int *, long
unsigned int *, _Bool)'}
lispbibl.d:13723:28: error: initializer element is not constant

Incidentally I found it necessary to reapply the patch to replace
token_type eof with token_type token_eof.

Any further info is much appreciated.

best,

Robert Dodier

Re: Build failure on Windows 10 with current version from Git

[Bruno H. <br...@cl...>]

Hi Robert,

> Here is the configure invocation:
> 
> MAKE=/c/msys64/mingw64/bin/mingw32-make.exe
> CC=/c/msys64/mingw64/bin/gcc.exe ./configure --with-debug
> --ignore-absence-of-libsigsegv --host=x86_64-w64-mingw32
> gcc.exe (Rev5, Built by MSYS2 project) 10.3.0

The file INSTALL.windows (at the top-level in the git repository)
lists the supported ways of building CLISP on Windows. MSYS is *not*
one of the options. But mingw is, with Cygwin as a host environment.

Bruno

Build failure on Windows 10 with current version from Git

[Robert D. <rob...@gm...>]

Hi, I'm trying to compile the current version (commit de01f0) on
Windows 10 and I'm running into compiler errors.

I don't expect detailed instructions but if someone has some hints
about how to solve the problems mentioned below, that would be very
helpful.

Here is the configure invocation:

MAKE=/c/msys64/mingw64/bin/mingw32-make.exe
CC=/c/msys64/mingw64/bin/gcc.exe ./configure --with-debug
--ignore-absence-of-libsigsegv --host=x86_64-w64-mingw32

gcc.exe --version reports:

gcc.exe (Rev5, Built by MSYS2 project) 10.3.0

Incidentally I'm working in a Git bash terminal.

Executing ming32-make.exe runs into some errors from gcc, namely that
token_type eof is an already-declared symbol of a different type; I
worked around that by changing token_type eof to token_type token_eof.
I can post a patch if anybody cares.

Beyond that, I run into two kinds of errors from gcc. (1) Apparently a
pointer won't fit into a long. I looked for a gcc flag to ensure long
is 64 bits but didn't find anything.

intparam.h:32:2: error: #error "Type char * does not fit into a long!!"
intparam.h:33:2: error: #error "Type long * does not fit into a long!!"
intparam.h:34:2: error: #error "Type function * does not fit into a long!!"

(2) Two other compiler errors.

spvw_mmap.d:306:11: error: conflicting types for 'mmap_prepare'
lispbibl.d:13723:28: error: initializer element is not constant

I feel like I could figure out how to work around (2), but (1) seems
like a show stopper. Does anyone know how to tell gcc that a long
should be 64 bits? (or whatever is needed to store a pointer) If
anyone knows how to fix (2) that would be great too.

Thanks a lot for any advice, I appreciate your help.

Robert Dodier

Re: Unexpected overflow in acosh and asinh

[Raymond T. <toy...@gm...>]

One more thing I just noticed.  (acosh 1d50) gives a division by 0.  But
asinh gives the expected answer.

On Mon, Jan 10, 2022 at 8:59 AM Raymond Toy <toy...@gm...> wrote:

> While working on some maxima stuff, I found out that clisp can't compute
> acosh(most-positive-double-float).  I get an overflow.  Same happens with
> asinh.  The actual answer should be about 710.475.
>
> I'm guessing this happens because clisp is using the log definition for
> these functions.  For example, asinh(x) = log(x+sqrt(x^2+1)).
>
> So for any value above sqrt(most-positive), there's an overflow.
>
> In this case, for large enough x, sqrt(x^2+1) = x so the final answer is
> log(2*x).  However, that would overflow if x is greater than
> most-positive/2.  In that case, we can use log(x)+log(x).
>
> This will allow clisp to compute acosh and asinh for the entire range of
> floats as expected.
>
> Oh, just noticed that clisp also fails to compute sinh(710.475d0).  This
> probably happens because sinh(x) is probably computed using
> (exp(x)-exp(-x))/2.  exp(710.475) overflows.  One possible way to avoid
> this is to compute exp(x/2)/sqrt(2).  This won't overflow.  Then square the
> result to get the final answer.
>
> --
> Ray
>


-- 
Ray

Unexpected overflow in acosh and asinh

[Raymond T. <toy...@gm...>]

While working on some maxima stuff, I found out that clisp can't compute
acosh(most-positive-double-float).  I get an overflow.  Same happens with
asinh.  The actual answer should be about 710.475.

I'm guessing this happens because clisp is using the log definition for
these functions.  For example, asinh(x) = log(x+sqrt(x^2+1)).

So for any value above sqrt(most-positive), there's an overflow.

In this case, for large enough x, sqrt(x^2+1) = x so the final answer is
log(2*x).  However, that would overflow if x is greater than
most-positive/2.  In that case, we can use log(x)+log(x).

This will allow clisp to compute acosh and asinh for the entire range of
floats as expected.

Oh, just noticed that clisp also fails to compute sinh(710.475d0).  This
probably happens because sinh(x) is probably computed using
(exp(x)-exp(-x))/2.  exp(710.475) overflows.  One possible way to avoid
this is to compute exp(x/2)/sqrt(2).  This won't overflow.  Then square the
result to get the final answer.

-- 
Ray

New French PO file for 'clisp' (version 2.49.60)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'clisp' has been submitted
by the French team of translators.  The file is available at:

    https://translationproject.org/latest/clisp/fr.po

(We can arrange things so that in the future such files are automatically
e-mailed to you when they arrive.  Ask at the address below if you want this.)

All other PO files for your package are available in:

    https://translationproject.org/latest/clisp/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    https://translationproject.org/domain/clisp.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

New French PO file for 'clisp' (version 2.49.60)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'clisp' has been submitted
by the French team of translators.  The file is available at:

    https://translationproject.org/latest/clisp/fr.po

(We can arrange things so that in the future such files are automatically
e-mailed to you when they arrive.  Ask at the address below if you want this.)

All other PO files for your package are available in:

    https://translationproject.org/latest/clisp/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    https://translationproject.org/domain/clisp.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

Re: Compiling on Apple Silicon machines

[Bruno H. <br...@cl...>]

Hi Roberto,

> I have downloaded the current sources (2.49.93+) and I have first
> compiled it with --enable-compatibility, and then I tried adding more
> modules and features.
> 
> Finally, I added
> 
>       # Mac OS X/arm64
>       darwin*--arm64)
>         XCFLAGS="$XCFLAGS -DNO_ASM"
>         ;;
> 
> to src/makemake.in (just after the corresponding block for Mac OS
> X/x86_64) and CLISP  compiles even with generational garbage
> collection.  It passes all tests except some socket related ones which
> seems to be known to give problems under macOS anyway also with intel
> CPUs. I was able to compile and run xindy.
> 
> My configuration command is
> 
> ./configure --with-libreadline-prefix=/opt/homebrew/opt/readline/            \
>             --with-libiconv-prefix=/opt/homebrew/opt/libiconv/               \
>             --with-module=libsvm --with-module=rawsock                       \
>             --with-module=pari --with-libpari-prefix=/opt/homebrew/opt/pari/ \
>             --cbc my-build-with-pari
> 
> - just to show which modules I load - and everything seems to run smoothly.

This is great news; thanks for sharing your experience and the recipe!

> I therefore suggest to perform the above addition to src/makemake.in

I followed the more detailed recipe from the file unix/PLATFORMS and
found that
  - all possible (non-experimental) configurations build fine and pass their
    tests (except for socket.tst, as you noted),
  - even the flag -DNO_ASM is not needed (most likely because the code
    so far has no assembly-language optimizations on arm64).

And the FFI is supported as well, through GNU libffcall version 2.4, which was
released two weeks ago.

I've committed the changes; the code from the git repository ('master'
branch) should now build out-of-the-box.

Bruno

Compiling on Apple Silicon machines

[Roberto A. <rob...@gm...>]

I have downloaded the current sources (2.49.93+) and I have first
compiled it with --enable-compatibility, and then I tried adding more
modules and features.

Finally, I added

      # Mac OS X/arm64
      darwin*--arm64)
        XCFLAGS="$XCFLAGS -DNO_ASM"
        ;;

to src/makemake.in (just after the corresponding block for Mac OS
X/x86_64) and CLISP  compiles even with generational garbage
collection.  It passes all tests except some socket related ones which
seems to be known to give problems under macOS anyway also with intel
CPUs. I was able to compile and run xindy.

My configuration command is

./configure --with-libreadline-prefix=/opt/homebrew/opt/readline/            \
            --with-libiconv-prefix=/opt/homebrew/opt/libiconv/               \
            --with-module=libsvm --with-module=rawsock                       \
            --with-module=pari --with-libpari-prefix=/opt/homebrew/opt/pari/ \
            --cbc my-build-with-pari

- just to show which modules I load - and everything seems to run smoothly.

I therefore suggest to perform the above addition to src/makemake.in

 Roberto

Clisp seems broken on s390x

[Christian E. <chr...@ca...>]

Hi,
I was starting to see current clisp in Ubuntu segfault when building Xindy
=> https://launchpad.net/ubuntu/+source/xindy/2.5.1.20160104-10

>From there I discovered that clisp itself is rather unhappy on s390x
(and ppc64el):
=> https://launchpad.net/ubuntu/+source/clisp/1:2.49.20180218+really2.49.92-3build5

The build fails like:
/home/ubuntu/clisp-2.49.20180218+really2.49.92/src/lispbibl.d:6673:6:
error: #error oint_addr_mask does not cover CODE_ADDRESS_RANGE !!
 6673 | #error oint_addr_mask does not cover CODE_ADDRESS_RANGE !!
      |

The code for the error in clisp is:

/* Verify the oint_addr_shift value w.r.t. the autoconfigured CODE_ADDRESS_RANGE
   and MALLOC_ADDRESS_RANGE values. */
#if !defined(WIDE_SOFT)
  /* The CODE_ADDRESS_RANGE needs to be checked because we store code
     pointers in Lisp objects (cf. macro ThePseudofun).
     In case of TYPECODES, the typecode() of such pointers must be machine_type,
     otherwise gc_mark() gets confused and crashes. */
  #if (CODE_ADDRESS_RANGE >> addr_shift) & ~(oint_addr_mask >> oint_addr_shift)
    #error oint_addr_mask does not cover CODE_ADDRESS_RANGE !!
  #endif

I debugged the values and they are:
CODE_ADDRESS_RANGE 0x000002AA09000000UL
addr_shift 0
oint_addr_mask (((2UL<<((64)-1))-1) & ~(0x01FC00000000UL | (1UL<<63)))
oint_addr_shift 0

So we can ignore the shft and this comes down to
  CODE_ADDRESS_RANGE & ~(oint_addr_mask)
  0x000002AA09000000UL & ~((((2UL<<((64)-1))-1) & ~(0x01FC00000000UL |
(1UL<<63))))
= 0x000002AA09000000UL & ~(FFFFFFFFFFFFFFFF & 7FFFFE03FFFFFFFF)
= 0x000002AA09000000UL & 800001FC00000000
= 000000A800000000

So the check is right, but I was unable to determine if the values made sense.
I checked former versions of clisp and tried to build it in all the
releases from 2018 until today.
But they all fail the same way.

For yet unknown reasons there was a single good build on 2019-09-05
=> https://launchpadlibrarian.net/440395396/buildlog_ubuntu-eoan-s390x.clisp_1%3A2.49.20180218+really2.49.92-3build3_BUILDING.txt.gz
But before&after and when retrying now all builds fail.

I was downloading the clean latest 2.49.92 and ran a standard configur&make.
It fails the same way, maybe that log can help you to spot the issue:
=> https://bugs.launchpad.net/ubuntu/+source/xindy/+bug/1931531/+attachment/5503834/+files/clsip-clean-fail.log

I've seen the list is rather inactive, but I didn't want to leave the
issue around without at least reporting it.

Thanks in advance if there are any hints on how to resolve this!

-- 
Christian Ehrhardt
Staff Engineer, Ubuntu Server
Canonical Ltd

New Serbian PO file for 'clisp' (version 2.49.60)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'clisp' has been submitted
by the Serbian team of translators.  The file is available at:

    https://translationproject.org/latest/clisp/sr.po

(We can arrange things so that in the future such files are automatically
e-mailed to you when they arrive.  Ask at the address below if you want this.)

All other PO files for your package are available in:

    https://translationproject.org/latest/clisp/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    https://translationproject.org/domain/clisp.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

Re: clisp release history

[Reini U. <rei...@gm...>]

Current ci status (all red for years):

https://ci.appveyor.com/project/rurban/clisp/history (windows)
https://travis-ci.org/github/rurban/clisp/builds (linux, osx)



Wolfgang Dautermann <wol...@da...> schrieb am So., 13. Sep. 2020,
13:16:

> Am 12.09.20 um 18:20 schrieb Pascal Bourguignon:
> > In days of git, indeed,  the notion of release may seem irrelevant.
> >
> > However, it’s a public relation device.
> > A release is something we write to make people aware of our product and
> > talk about it, and forget about the competition.
>
> Hi,
> yes - and often a binary (e.g. for Windows, MacOS, etc. is *really*
> helpful.
>
> It seems, that you are working with MinGW (the last commit on
> https://gitlab.com/gnu-clisp/clisp/-/commits/master
> is 'Fix a build error on mingw.', so it seems to work. (at least, a
> native build on Windows, I assume, that is not a crosscompiled build?)
>
> An installer or zip file including the compiled Windows binary would be
> really helpful (maybe one for 32 bit and one for 64 bit).
>
> I use the latest Windows release (yes, that from 2010!) for
> crosscompiling the open source CAS Maxima for Windows (using 'Wine').
>
> Now other Lisps are supported too (SBCL, ABCL), but when I started the
> idea, Clisp was the first Lisp compiler which worked. And it is still
> included.
>
> I have some code to crosscompile (using Linux as build host) Clisp (and
> the required libraries) for Windows, but it does not work now.
>
> Best regards, Wolfgang
>
>
> _______________________________________________
> clisp-devel mailing list
> cli...@li...
> https://lists.sourceforge.net/lists/listinfo/clisp-devel
>

Re: clisp release history

[Wolfgang D. <wol...@da...>]

Am 12.09.20 um 18:20 schrieb Pascal Bourguignon:
> In days of git, indeed,  the notion of release may seem irrelevant.
> 
> However, it’s a public relation device. 
> A release is something we write to make people aware of our product and
> talk about it, and forget about the competition.

Hi,
yes - and often a binary (e.g. for Windows, MacOS, etc. is *really* helpful.

It seems, that you are working with MinGW (the last commit on
https://gitlab.com/gnu-clisp/clisp/-/commits/master
is 'Fix a build error on mingw.', so it seems to work. (at least, a
native build on Windows, I assume, that is not a crosscompiled build?)

An installer or zip file including the compiled Windows binary would be
really helpful (maybe one for 32 bit and one for 64 bit).

I use the latest Windows release (yes, that from 2010!) for
crosscompiling the open source CAS Maxima for Windows (using 'Wine').

Now other Lisps are supported too (SBCL, ABCL), but when I started the
idea, Clisp was the first Lisp compiler which worked. And it is still
included.

I have some code to crosscompile (using Linux as build host) Clisp (and
the required libraries) for Windows, but it does not work now.

Best regards, Wolfgang

Re: clisp release history

[Pascal B. <pj...@in...>]

> On 12 Sep 2020, at 18:12, Don Cohen <don...@is...> wrote:
> 
> Someone pointed out to me recently that the release history,
> which shows up at the top of impnotes, shows as the last release:
>  Release 2.49  2010-07-07 sds
> 
> This gave him the impression that there has been no development or
> maintenance for the last 10 years!
> 
> I don't think I understand what exactly qualifies as a "release" 
> or how this is determined.  If there are not to be any more
> official releases, I suggest that some explanation of that replace
> the list of releases, or be added to the list of releases.
> And if there actually were, or even SHOULD have been releases in 
> the past 10 years, that those be added to the list.

In days of git, indeed,  the notion of release may seem irrelevant.

However, it’s a public relation device. 
A release is something we write to make people aware of our product and talk about it, and forget about the competition.

Some people don’t even care giving release numbers to their products, they just use the year and month of the release, and they make periodic releases (monthly, quaterly, etc).

Perhaps we could write a release note generation program that would extract git commit messages and format them nicely to issue quarterly releases?



-- 
__Pascal J. Bourguignon__

clisp release history

[<don...@is...>]

Someone pointed out to me recently that the release history,
which shows up at the top of impnotes, shows as the last release:
  Release 2.49  2010-07-07 sds

This gave him the impression that there has been no development or
maintenance for the last 10 years!

I don't think I understand what exactly qualifies as a "release" 
or how this is determined.  If there are not to be any more
official releases, I suggest that some explanation of that replace
the list of releases, or be added to the list of releases.
And if there actually were, or even SHOULD have been releases in 
the past 10 years, that those be added to the list.

AW: [clisp:feature-requests] #57 Solaris 11/sparc 64bit

[<Joe...@t-...>]

Hi,

That's not so much a feature request as a bug report (actually 2 of them).

>Cannot map memory to address 0x17000000000000 .
>[spvw_mmap.d:347] errno = ENOMEM: Not enough memory.
CLISP did not expect this memory map.
Try and build CLISP using one of the many other memory model configurations (or use the make target that builds a dozen different configurations), then please report here which one of these works for you.
https://clisp.sourceforge.io/impnotes/memory-models.html
Among these, there are memory models that should work robustly with any unforeseen OS memory map (esp. among HEAPCODES_xyz rather than TYPECODES), albeit at the cost of being very slow. Yet that might be enough for you, depending on your needs.

If you can dig into technical levels, have a look at the top of lispbibl.d about these choices and further down about build-porting64-gcc-portability, address space layout and *_ADDRESS_RANGE. Perhaps something very simple is broken (or not up to date), given that it does not recognize "sparcv9" instead of "sparc64"?

Are you actually using online clisp sources or an ancient released version?

Regards,
	Jörg Höhle

-----Original Message-----
Von: Joern Clausen <joe...@us...> 
Gesendet: Dienstag, 1. September 2020 08:59
An: Ticket #57: Solaris 11/sparc 64bit <57...@fe...>
Subject: [clisp:feature-requests] #57 Solaris 11/sparc 64bit

** [feature-requests:#57] Solaris 11/sparc 64bit**
**Status:** open
**Group:**
**Created:** Tue Sep 01, 2020 06:59 AM UTC by Joern Clausen **Last Updated:** Tue Sep 01, 2020 06:59 AM UTC
**Owner:** nobody

Building CLISP on Solaris 11.4 on a Sparc (sparcv9) 64-bit processor using GCC 9.2.0 fails with:

~~~
gcc -I/opt/pkg-hrz/20200817/include -I/opt/pkg-hrz/20200817/include/ncurses -I/opt/pkg-hrz/20200817/include/glib-2.0 -I/opt/pkg-hrz/20200817/include/gio-unix-2.0 -I/opt/pkg-hrz/20200817/lib/glib-2.0/include -I/usr/include  -I/opt/pkgsrc/pkg-hrz/pkgsrc/lang/clisp/work.pkgsrc-sol11sparc/clisp-2.49/src/gllib -O2 -D_FORTIFY_SOURCE=2 -I/opt/pkg-hrz/20200817/include -I/opt/pkg-hrz/20200817/include/ncurses -I/opt/pkg-hrz/20200817/include/glib-2.0 -I/opt/pkg-hrz/20200817/include/gio-unix-2.0 -I/opt/pkg-hrz/20200817/lib/glib-2.0/include -I/usr/include -W -Wswitch -Wcomment -Wpointer-arith -Wimplicit -Wreturn-type -Wmissing-declarations -O2 -DUNIX_BINARY_DISTRIB -DENABLE_UNICODE -DDYNAMIC_FFI -DDYNAMIC_MODULES -I. -c lisparit.c
In file included from lisparit.d:28:
arilev1.d:251:18: fatal error: arisparc64.c: No such file or directory
  251 |         #include "arisparc64.c"
      |                  ^~~~~~~~~~~~~~
compilation terminated.
~~~

Fixing makemake to check for "sparcv9" instead of "sparc64", arisparc64.c is generated and compilation succeeds, but lisp.run is not useable:

~~~
$ lisp.run
Cannot map memory to address 0x17000000000000 .
[spvw_mmap.d:347] errno = ENOMEM: Not enough memory.
Trying to make room through a GC...
Cannot map memory to address 0x17000000000000 .
[spvw_mmap.d:347] errno = ENOMEM: Not enough memory.
Trying to make room through a GC...
Cannot map memory to address 0xb000000000000 .
[spvw_mmap.d:347] errno = ENOMEM: Not enough memory.
Trying to make room through a GC...
...
~~~

AW: thread pools?

[<Joe...@t-...>]

Hi Don,

>Is there some existing implementation of thread pools for clisp?
Not that I've heard of.

I would *believe* the MT authors did not think about pools and that's why there is no mention of them in impnotes.

Conceptually, using or not using pools could be just a *use-thread-pool* choice. However, one should be aware of possible consequences. A handle to a closed socket or closed file should become unusable. But do you remember that ancient bug where CLISP closed the wrong socket fd?
A handle to a "job done" thread should become unusable as well, one would expect IMHO. But if that thread lives on in a pool, wouldn't the handle survive as well? When would you expect FINALIZE to be called?

The only nice solution appears to dissociate the Lisp handle from the OS thread handle. Then, on the Lisp level, FINALIZE and THREAD-INTERRUPT can only ever work on the original thread and no buggy code would be able to send spurious signals to the reused thread.

>Perhaps threads to be used in a pool should always be doing something, like (loop (sleep 1e6))?
That's basically how pools work, except they dispense with a timeout and simply pause/wait forever, i.e. until a signal (job request) kicks in.
Note that expecting THREAD-INTERRUPT to be able to submit a new request to a (hopefully) sleeping thread is a design decision that is at odds with the design I sketched above. Yes, there are different solution paths, with pros and cons.

>Would it be much faster to use an existing thread from a  pool than to create a new one? 
Much? I don't know. However that's typically the reason why they are used and reused.

>Should I write my own?
Only you can tell ;-)
Basically, if THREAD-INTERRUPT works well, you could write your own at Lisp-level, based on the existing primitives. You would have to provide your own THREAD-ACTIVE-P, THREAD-JOIN (perhaps based on EXEMPTION-WAIT?) and possibly more. FINALIZE on a thread object would not be called as early as before.
If you completely wrap all MT:THREAD objects, then you will be able to prevent rogue code from sending interrupts to reused threads, as sketched above. It might be unwieldly, e.g. MT:MUTEX-OWNER returns a thread object; would you wrap that too? Or simply live with the limitations?
Wrap and wrap and wrap and adding another pointer indirection, is that what 70 years of computer science have taught us?

BTW, I see 2 bugs in https://clisp.sourceforge.io/impnotes/mt.html
1. (MT:MUTEX-NAME thread) should operate on a mutex object instead. 
2. (MT:EXEMPTION-NAME thread) should operate on an exemption object instead.

Regards,
	Jörg

thread pools?

[<don...@is...>]

Is there some existing implementation of thread pools for clisp?
Would it be much faster to use an existing thread from a  pool 
than to create a new one? 
Should I write my own?
It's not clear to me from impnotes what happens to a thread that 
finishes.  I guess if it's GC-able then it gets GC'd, but if not,
I hope it incurs no cost other than its own storage?  
And when it is finished can I get it to do something else by 
calling thread-interrupt on it?
Or does this only work with non-terminated threads?
And is a thread that finishes its function automatically "terminated"?

Perhaps threads to be used in a pool should always be doing 
something, like (loop (sleep 1e6))?  And then I can get them to 
do something else with thread-interrupt, right?  After which they
return to what they were doing when interrupted, right?  So they
can be interrupted to do something else, right?

Is this all supposed to be obvious, or should impnotes say 
something about it?

Re: stack overflow => segfault in MT

[<don...@is...>]

Picking up from a thread from 2011 (!!), Vladimir Tzankov wrote:

  With MT - only the "main thread" (first one) has stack overflow
  detection.  Stack overflow detection/recovery is implemented via
  libsigsegv and it supports single stack range.

With current clisp source I have an example in which non-MT detects a
stack overflow whereas in MT, the main thread, having started no 
other threads, still segfaults.
When I try simple infinite recursions I do get stack overflow
errors that do not exit lisp.

I notice when I trace a function in the recursion, it gets about
twice as deep (41 calls) before segfault in MT as before stack
overflow in non-MT (21 calls).

What would be useful debugging information and how can I get it?

more on lisp-implementation-version

[<don...@is...>]

First, in spite of no changes to soure, the nightly build
last night did not reproduce the error I reported yesterday.
The doc in impnotes:

 25.3.4. Function LISP-IMPLEMENTATION-VERSION

 LISP-IMPLEMENTATION-VERSION returns the numeric version (like 3.14),
 and the release date (like "1999-07-21"). When running on the same
 machine on which CLISP was built, it appends the binary build and
 memory image dump date in universal time (like 3141592654). When
 running on a different machine, it appends the MACHINE-INSTANCE of
 the machine on which it was built.

One question is how it decides whether running on the same machine.
Another is where it gets this thing that looks like an ip address
"2.49.93+ (2018-02-18) (built on number15.don-eve [198.105.244.228])"
I don't believe that this was ever the ip address of this machine.

nightly build failure (fedora 24)

[<don...@is...>]

==> cbcstep3.log <==
Test failed:
-rw-r--r--. 1 don don 899 Nov  6 03:05 socket.erg
To see which tests failed, type
    cat /home/don/clisp/build-dir/tests/*.erg
Makefile:34: recipe for target 'compare' failed
make[1]: *** [compare] Error 1
make[1]: Leaving directory '/home/don/clisp/build-dir/tests'
Makefile:2218: recipe for target 'check-tests' failed
make: *** [check-tests] Error 2
===> make check FAILED


... cat /home/don/clisp/build-dir/tests/socket.erg
Form: (MULTIPLE-VALUE-BIND (RUN ARGS) (CMD-ARGS) (LET ((IS (RUN-PROGRAM RUN :ARGUMENTS (APPEND ARGS '("-q" "-q" "-x" "
            (let ((se (socket:socket-server)))
              (write-line (princ-to-string (socket:socket-server-port se)))
              (with-open-stream (so (socket:socket-accept se))
                (write-line (lisp-implementation-version) so))
              (socket:socket-server-close se))
")) :INPUT NIL :OUTPUT :STREAM)) LOCAL REMOTE) (LOOP :UNTIL (DIGIT-CHAR-P (PEEK-CHAR NIL IS)) :DO (READ-LINE IS)) (WITH-OPEN-STREAM (SO (SOCKET-CONNECT (READ IS) "localhost" :TIMEOUT 10)) (OR (STRING= (SETQ LOCAL (LISP-IMPLEMENTATION-VERSION)) (SETQ REMOTE (READ-LINE SO))) (LIST :LOCAL LOCAL :REMOTE REMOTE)))))
CORRECT: T
CLISP  : (:LOCAL "2.49.93+ (2018-02-18) (built 3782026988) (memory 3782027047)" :REMOTE "2.49.93+ (2018-02-18) (built on number15.don-eve [23.217.138.107])")

Re: [PATCH] Fix broken defsetf for places with keywords.

[Bruno H. <br...@cl...>]

Sam Steingold wrote:
> Sorry about not replying earlier - I thought that you were actually
> planning to work more on your patch.

Very good point. My initial reaction to Kaz' patch in
<https://sourceforge.net/p/clisp/mailman/message/36601264/>
was "who needs the generic case, the frequent case is that the keywords
are known at compile time". The only thing I did not like about it was
the (not (constantp ev-argform)) - the error checking of the keywords
should be done by DEFSETF, not in GET-SETF-EXPANSION.

But you are absolutely right: When writing compilers (and macroexpansion
code), one should strive for maximum generality, and the optimizations
of common cases are only special cases.

So, looking at
http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/sec_5-1-1-2.html
http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/fun_get-setf-expansion.html
http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/mac_defsetf.html
there is even an example regarding computed keywords:

(defsetf xy (&key ((x x) 0) ((y y) 0)) (store)
  `(set-xy ,store 'x ,x 'y ,y)) =>  XY
(get-setf-expansion '(xy a b))
=> (#:t0 #:t1),
   (a b),
   (#:store),
   ((lambda (&key ((x #:x)) ((y #:y))) 
      (set-xy #:store 'x #:x 'y #:y))
    #:t0 #:t1),
   (xy #:t0 #:t1)

This example gives an important clue: How to use a LAMBDA for keyword matching.
But this example is wrong. It ignores the keyword init forms. The correct
result is:

(get-setf-expansion '(xy a b))
=> (#:t0 #:t1),
   (a b),
   (#:store),
   ((lambda (&key ((x #:x)) ((y #:y))) 
      (set-xy #:store 'x #:x 'y #:y))
    #:t0 #:t1 'x 0 'y 0),
   (xy #:t0 #:t1 'x 0 'y 0)
or
=> (#:t0 #:t1),
   (a b),
   (#:store),
   ((lambda (&key ((x #:x) 0) ((y #:y) 0)) 
      (set-xy #:store 'x #:x 'y #:y))
    #:t0 #:t1),
   (xy #:t0 #:t1 'x 0 'y 0)

Things get more hairy when we think of keyword init forms that are
not constant forms. In which environment do they get evaluated?

Example:
(defun zero () 0)
(flet ((default-form-for-y () '(zero)))
  (defsetf xy (&key ((x x) '(zero)) ((y y) (default-form-for-y))) (store)
    `(set-xy ,store 'x ,x 'y ,y))) =>  XY
There are two evaluations:
  1. At macro expansion time, i.e. during get-setf-expansion,
     (default-form-for-y) gets evaluated, in the lexical environment of the DEFSETF form.
     => (ZERO).
  2. At runtime, (ZERO) gets evaluated. => 0.

So, DEFSETF needs to store a closure that will evaluate
(default-form-for-y).

The other thing is: How should the desired result of GET-SETF-EXPANSION
look like? Let's take an example with keywords arguments that are not in
the keyword package, non-constant keyword init forms, and with a required
argument as well.

> (defsetf xy (r &key ((x x) '(zero)) ((y y) '(zero))) (store) `(set-xy ,store ,r 'x ,x 'y ,y))
XY

The simple case, when all keywords are specified as constants, is:

> (get-setf-expansion '(xy 3 'x 0 'y 0))
(#:G1 #:G2 #:G3) ;
(3 0 0) ;
(#:G4) ;
(SET-XY #:G4 #:G1 'X #:G2 'Y #:G3) ;
(XY #:G1 'X #:G2 'Y #:G3)

The general case, when all keyword arguments are non-constant, requires
the LAMBDA trick, and the mechanism for using the keyword init form
shown above:

> (get-setf-expansion '(xy 3 a 0))
(#:G1 #:G2 #:G3 #:G4 #:G5) ;
(3 A 0 (ZERO) (ZERO)) ;
(#:G6) ;
((LAMBDA (&KEY ((X #:G7)) ((Y #:G8))) (SET-XY #:G6 #:G1 'X #:G7 'Y #:G8)) #:G2 #:G3 'X #:G4 'Y #:G5) ;
(XY #:G1 #:G2 #:G3 'X #:G4 'Y #:G5)

In the mixed case, when some of the keyword arguments are constant,
the corresponding expansion can be optimized to omit the init forms
that are not needed:

> (get-setf-expansion '(xy 3 a 0 'y 0))
(#:G1 #:G2 #:G3 #:G4 #:G5) ;
(3 A 0 0 (ZERO)) ;
(#:G6) ;
((LAMBDA (&KEY ((X #:G7)) ((Y #:G8))) (SET-XY #:G6 #:G1 'X #:G7 'Y #:G8)) #:G2 #:G3 'Y #:G4 'X #:G5) ;
(XY #:G1 #:G2 #:G3 'Y #:G4 'X #:G5)

This was the hard part. Actually implementing it was easy, then.
Done and pushed (including your test case).

Bruno