clisp-devel — closed list for developers and testers

Re: on cygwin target, executable only starts if named lisp.exe

[Vibhu M. <vi...@ho...>]

The configure flag --without-dynamic-modules solves the executable image 
saving/loading problem.

----

Details:

src/lispbibl.d contains this code below, which I'm guessing causes an 
import table entitled "lisp.exe" and containing function names to be placed 
in base/lisp.exe. Such a title is apparently what causes a problem if the 
executable file is renamed to something else or saved as an executable 
image named other than "lisp.exe". Disabling dynamic modules prevents such 
a table from being produced and embedded, allowing (saveinitmem "test2" 
:executable t) to create a successfully startable file named other than 
"lisp.exe".

Extract from src/lispbibl.d:

/* this is necessary to avoid messages like
    Info: resolving _mv_space by linking to __imp__mv_space (auto-import)
    on woe32. */
#if defined(DYNAMIC_MODULES)
   #if defined(WIN32_NATIVE) || defined(UNIX_CYGWIN)
     #define DYNAMIC_TABLES 1
     #define modexp  __declspec(dllexport)
     #define modimp  __declspec(dllimport)
     #define EXECUTABLE_NAME "lisp.exe"
   #else
     #define DYNAMIC_TABLES 0
     #define modexp
     #define modimp  extern
   #endif
#else
   #define DYNAMIC_TABLES 0
   #define modexp
   #define modimp  extern
#endif

Cross-check: I notice that indeed (software-type):
1. in my earlier problematic cygwin target build shows the 
-DDYNAMIC_MODULES flag.
2. in the new build described above doesn't show that flag.
3. in the old mingw target build that didn't exhibit the problem, doesn't 
show that flag, so it must have been turned off automatically for mingw.

So it does look like my theory above is correct.

Also, the clisp packaged with cygwin does show -DDYNAMIC_MODULES in 
(software-type), which may be why its builder patched his build to disable 
saving executable images.

Vibhu

Re: Cygwin targetting mingw 32-bit

[Vibhu M. <vi...@ho...>]

Below, I'm going to round out an email exchange from a year ago based on 
something I've just found.

On 27/09/2022 06:28, Vibhu Mohindra wrote:
> On 27/09/2022 01:59, Bruno Haible wrote:
>> Vibhu Mohindra wrote:
>>> System: Windows 10 64-bit, 6 GB RAM
>>> Processor: Intel Xeon X5647, x64-based processor
>>> Cygwin (64-bit) targetting mingw 32-bit
>>> clisp revision: f1aa42219433f019c05e707220538b3634bc233c
>>> I've used the instructions in INSTALL.windows.
>>>
>>> ----
>>> 1. LN_S
>>>
>>> The Makefile has a variable LN_S that is chosen to be some platform
>>> dependent value.
>>
>> src/makemake.in says that valid values for this variable include
>> 'ln -s', 'ln', and 'cp -p'.
>>
>>> The reason is that build-aux is a directory, and hard linking
>>> directories is not supported.
>>
>> I don't think it's good for the Makefile to copy an entire build-aux
>> directory. We should look into how to avoid that. Where is this
>> happening?
> 
> Oh right. If LN_S is never meant to be applied to directories, then that's 
> OK, and it's simply a matter of identifying where all it's used for 
> directories.
> 
> The top level Makefile in the build directory (from one of my 
> configurations) has this for the build-aux target.
> 
> build-aux : ../src/build-aux
>      -$(RM) build-aux
>      -$(LN_S) ../src/build-aux build-aux
> 
> A quick check with grep doesn't reveal any other places where LN_S is used 
> for directories. Anyway, now that I know the intention, if I come across it 
> being used anywhere else for directories, I'll fix that specific instance 
> and write in, rather than trying to change the definition of LN_S itself.
> 
> If build-aux gets fixed in GitLab, I'll immediately pull that down.
> 
>>> Does "cp -a" seem a reasonable choice?
>>
>> Yes, if directories need to be copied as well, then 'ln' and 'cp -p'
>> won't work and 'cp -a' is better.

I'm sorry, my ln -s woes seem to have been because I had the clisp tree on 
a network drive mapped to Z:, which doesn't support hard links and whose 
soft links are impenetrable to ld. With the tree on a local drive (C:), I 
don't seem to need to change anything related to LN_S from its defaults in 
Git. Terribly sorry about the LN_S report. (Other notes I've written are 
all still valid.)

Vibhu

patch for mingw target

[Vibhu M. <vi...@ho...>]

I've been using the attached patch when building in Cygwin 32-bit 
targetting Mingw 32-bit target. It gets past the make error shown in the 
attached make.log by renaming the eof symbol to something else (token_eof).

Vibhu

Re: on cygwin target, executable only starts if named lisp.exe

[Vibhu M. <vi...@ho...>]

On 05/04/2024 02:52, Vibhu Mohindra wrote:
> If anyone makes observations that contradict mine, I'd love to hear, 
> because it means I should start afresh and rationalise my build software 
> versions and configuration options. I'm posting this in case it's a real 
> bug easily reproducible with whatever Cygwin version and configuration 
> flags anyone may use.

I've verified that the problem also occurs on a fresh checkout of clisp 
head, using the latest Cygwin and Cygwin packages, this time with 
libsigsegv libs/headers installed.

./configure build3c64c64
configure results: FFI: yes, readline: no, libsigsegv: yes
build succeeds, out of the box.
loading a saved executable image has the same problem, segfault.

I also notice that Cygwin has a clisp package of its own. That build has a 
patch dated April 2015 applied to refuse to save images if they are to be 
executable. Maybe the patch author noticed the same problem back then that 
I have noticed now.

https://cygwin.com/cgit/cygwin-packages/clisp/tree/cygwin-no-save-executable.patch

Vibhu

mingw target pathname error

[Vibhu M. <vi...@ho...>]

In September 2022, Bruno had mentioned some known file/filesystem trouble 
if building from Cygwin 64-bit targetting Cygwin 32-bit. I don't know if 
that's related to some observations I've made, so I'll report my 
observations anyway because they might be totally unrelated.

The problem occurs on both:
1) 64-bit Windows, 32-bit Cygwin, targetting 32-bit Mingw
2) 32-bit Windows, 32-bit Cygwin, targetting 32-bit Mingw
and does not occur in any of the following places I've tested targetting 
Cygwin:
3) 64-bit Windows, 64-bit Cygwin, targetting 64-bit Cygwin
4) 64-bit Windows, 32-bit Cygwin, targetting 32-bit Cygwin

This is the problem.

Z:\clisp_20240322\builds\tryc32m32>lisp -M lispinit.mem -norc -ansi -q
[1]> (directory (make-pathname :type :wild :name :wild))

*** - Internal error: statement in file "../src/pathname.d", line 3769 has 
been reached!!
       Please see <http://clisp.org/impnotes/faq.html#faq-bugs> for bug 
reporting instructions.
The following restarts are available:
ABORT          :R1      Abort main loop
Break 1 [2]> :a
[3]>

Without the -norc, the error occurs even before the first clisp prompt, 
presumably from trying to look up the rc file.

Git revision:
66924971790e4cbee3d58f36e530caa0ad568e5f (July 2023)

Vibhu

on cygwin target, executable only starts if named lisp.exe

[Vibhu M. <vi...@ho...>]

I observe a particular problem on my builds targetting Cygwin but not on my 
builds targetting Mingw.

----

The problem occurs on:
a 64-bit Windows host building
- in Cygwin 64-bit for Cygwin 64-bit
and
- in Cygwin 32-bit for Cygwin 32-bit
but does not occur:
- in Cygwin 32-bit for Mingw 32-bit
or on a 32-bit Windows host building
- in Cygwin 32-bit for Mingw 32-bit

----

The problem is this. The build produces a (lisp.exe, lispinit.mem) pair in, 
say, build1/ and build1/base/. I'll call these core and base respectively. 
Running core, I can do:
(saveinitmem "test2" :executable t)
and get a test2.exe that starts successfully.
But if I run base and save a memory image in the same fashion, I can't 
start the resulting test2.exe. Trying to start it produces a complaint 
about a missing library lisp.exe.

----

Here are some observations, each described independently of all the others.

1) If I save the image with the name "lisp", or otherwise just rename the 
created image file to lisp.exe, then it starts without crashing.

2)
cd build1/base
mv lisp.exe lisp2.exe
./lisp2.exe -M lispinit.mem
fails to start in the same way.
So apparently, the executable must be called lisp.exe.
[core does not have this problem]

3)
ldd build1/base/lisp.exe
produces a list of dlls. But if I rename lisp.exe to e.g. test2.exe, then
ldd build1/base/test2.exe
produces the same list with one extra item in it: "lisp.exe", which is 
unresolved.
[core does not have this problem and shows a single set of dlls regardless 
of the executable's name, and the set never includes "lisp.exe" or the 
executable's name]

4)
strings build1/base/lisp.exe
reveals three occurrences of "lisp.exe". Using a binary editor to replace 
the third one and its trailing zeroes with a name of my choice and trailing 
zeroes, and then renaming the file to have that same name of choice seems 
to solve the problem. The executable then does start without crashing. This 
third occurrence of "lisp.exe" is embedded in a list of what look like dll 
dependency names.
[core only has the first two occurrences of "lisp.exe"]

----

I'm on latest GitLab head:
66924971790e4cbee3d58f36e530caa0ad568e5f (July 2023)
but I believe the problem exists even on an older tree I have
f1aa42219433f019c05e707220538b3634bc233c (September 2022)

The builds above do not use libsigsegv.

----

If anyone makes observations that contradict mine, I'd love to hear, 
because it means I should start afresh and rationalise my build software 
versions and configuration options. I'm posting this in case it's a real 
bug easily reproducible with whatever Cygwin version and configuration 
flags anyone may use.

Thanks in advance,

Vibhu

New French PO file for 'clisp' (version 2.49.60)

[Translation P. R. <ro...@tr...>]

Hello, gentle maintainer.

This is a message from the Translation Project robot.

A revised PO file for textual domain 'clisp' has been submitted
by the French team of translators.  The file is available at:

    https://translationproject.org/latest/clisp/fr.po

(We can arrange things so that in the future such files are automatically
e-mailed to you when they arrive.  Ask at the address below if you want this.)

All other PO files for your package are available in:

    https://translationproject.org/latest/clisp/

Please consider including all of these in your next release, whether
official or a pretest.

Whenever you have a new distribution with a new version number ready,
containing a newer POT file, please send the URL of that distribution
tarball to the address below.  The tarball may be just a pretest or a
snapshot, it does not even have to compile.  It is just used by the
translators when they need some extra translation context.

The following HTML page has been updated:

    https://translationproject.org/domain/clisp.html

If any question arises, please contact the translation coordinator.

Thank you for all your work,

                                The Translation Project robot, in the
                                name of your translation coordinator.
                                <coo...@tr...>

Re: formatter bug?

[Bruno H. <br...@cl...>]

Vibhu Mohindra wrote:
> (defun s (fillp)
>    (make-array 1 :element-type 'character :initial-contents "x"
>                :fill-pointer fillp))
> (defun fmt (fillp)
>    (format nil (eval `(formatter ,(s fillp)))))
> (fmt nil) ;ok
> (fmt t)   ;error due to base-string instead of simple-string
> ...
> ----
> http://www.lispworks.com/documentation/HyperSpec/Body/m_format.htm#formatter
> says the formatter macro's arg is a "format string".
> http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_f.htm#format_string
> defines "format string" to be a string and doesn't restrict it to any 
> particular subtype.

Thanks for the report. Registered at https://gitlab.com/gnu-clisp/clisp/-/issues/37
and fixed in git.

Bruno

formatter bug?

[Vibhu M. <vi...@ho...>]

(defun s (fillp)
   (make-array 1 :element-type 'character :initial-contents "x"
               :fill-pointer fillp))
(defun fmt (fillp)
   (format nil (eval `(formatter ,(s fillp)))))
(fmt nil) ;ok
(fmt t)   ;error due to base-string instead of simple-string
#|
CLISP: fails
CCL:   fails
CMUCL: passes
SBCL:  passes
ECL:   passes
ABCL:  passes
|#

----
http://www.lispworks.com/documentation/HyperSpec/Body/m_format.htm#formatter
says the formatter macro's arg is a "format string".
http://www.lispworks.com/documentation/HyperSpec/Body/26_glo_f.htm#format_string
defines "format string" to be a string and doesn't restrict it to any 
particular subtype.


-- 
Vibhu

Re: REPL bug

[Bruno H. <br...@cl...>]

Vibhu Mohindra wrote:
> $ clisp -q -ansi -norc
> [1]> ; a comment line makes repl ignore second expr on following line
> nil (format t "~&never printed~%")
> NIL
> [2]>
> (+ 2 2)
> 4
> [3]>

This looks like a bug; I don't have an immediate explanation for this
strange behaviour. I've now registered it at
https://gitlab.com/gnu-clisp/clisp/-/issues/36

Thanks for having reported it!

Bruno

REPL bug

[Vibhu M. <vi...@ho...>]

$ clisp -q -ansi -norc
[1]> ; a comment line makes repl ignore second expr on following line
nil (format t "~&never printed~%")
NIL
[2]>
(+ 2 2)
4
[3]>

$ clisp --version
GNU CLISP 2.49.93+ (2018-02-18) (built on x86-csail-01.debian.org 
[128.31.0.50])
Software: GNU C 10.3.0
gcc -g -O2 
-ffile-prefix-map=/build/clisp-E4DSi0/clisp-2.49.20210628.gitde01f0f=. 
-fstack-protector --param=ssp-buffer-size=4 -Wformat 
-Werror=format-security -no-integrated-cpp -W -Wswitch -Wcomment 
-Wpointer-arith -Wreturn-type -Wmissing-declarations -Wimplicit 
-Wno-sign-compare -Wno-format-nonliteral -Wno-shift-negative-value -O 
-fwrapv -fno-strict-aliasing -DNO_ASM -DENABLE_UNICODE -DDYNAMIC_FFI 
-DDYNAMIC_MODULES -Wl,-z,relro  libgnu.a  -lreadline -lncurses -ldl 
-lffcall  -lsigsegv -lunistring
SAFETY=0 HEAPCODES ONE_FREE_BIT_HEAPCODES WIDE_HARD SPVW_BLOCKS 
SPVW_MIXED TRIVIALMAP_MEMORY
libsigsegv 2.13
libreadline 8.1
libffcall 2.4
Features:
(READLINE REGEXP WILDCARD SYSCALLS I18N LOOP COMPILER CLOS MOP CLISP ANSI-CL
  COMMON-LISP LISP=CL INTERPRETER LOGICAL-PATHNAMES SOCKETS GENERIC-STREAMS
  SCREEN FFI GETTEXT UNICODE BASE-CHAR=CHARACTER WORD-SIZE=64 PC386 UNIX)
C Modules: (clisp i18n syscalls regexp readline)
Installation directory: /usr/lib/clisp-2.49.93+/
User language: ENGLISH
Machine: X86_64 (X86_64) localhost.localdomain [127.0.1.1]


----
I had meant to look at the code to see if I could fix this, but half a 
year has passed and I realise I still haven't. So I suppose I should at 
least report it to this list.


Vibhu

Re: Cygwin targetting mingw 32-bit

[Vibhu M. <vi...@ho...>]

On 27/09/2022 00:18, Vibhu Mohindra wrote:
> 1. LN_S

I've got past all "ln -s" related problems locally and will describe my 
understanding of things.

System: Windows 10 64-bit
Cygwin (32-bit) targetting mingw 32-bit


----
I changed my toplevel Makefile's build-aux target to use "cp -a" instead 
of $(LN_S), because LN_S is automatically set to "ln" in this Makefile 
and "ln" doesn't work for directories.


----
There are two more kinds of occurrences of ln -s that need fixing.

1) In Makefiles in subdirectories like build/{gllib,i18n,...}/Makefile
2) clisp-link (and perhaps other scripts) which cause full/* to be symlinks

1)

Here's how LN_S gets its value in the Makefiles. The configure script 
detects in Cygwin that "ln -s" works and sets LN_S to that. It can't 
know that the Windows native clisp that will be produced via mingw won't 
actually be able to handle symlinks. (Presumably no native Windows 
programs can handle symlinks.) However src/makemake.in (near line 1052) 
does know that fact so it replaces LN_S to be "ln". In short, configure 
thinks "ln -s" but makemake.in knows to override that to "ln".

But the subsidiary Makefiles take their LN_S values directly from 
configure not from makemake.in. Thus they get the configure-detected bad 
value "ln -s". I manually changed the LN_S values in these subsidiary 
Makefiles to be "ln". I don't know how the build architecture should 
change so that they don't require manual editing but are instead born 
with the correct value just as the toplevel Makefile is. (Maybe passing 
LN_S=ln as a command-line argument to configure will do the trick. If 
so, then the solution requiring the least work will be to update 
INSTALL.windows to state that. But this hack will still leave the 
build's design flawed -- where there's the appearance that makemake gets 
to have a say when in reality it doesn't consistently.)

2)

I've got "make full" to complete but I couldn't immediately run the 
result. Again, that's because full/* were mostly symlinks. Changing 
lisp.exe and lispinit.mem to instead be hard links of the underlying 
files made "clisp -K full" start up successfully. I think the symlinks 
were created due to clisp-link having "ln -s" in two places. These two 
places (or whichever other scripts caused full/* to be symlinks) should 
be changed.


-- 
Vibhu

Re: Cygwin targetting mingw 32-bit

[Vibhu M. <vi...@ho...>]

On 02/10/2022 04:42, Vibhu Mohindra wrote:
> To link the object files into an executable, -lbcrypt is needed but 
> missing.

To put that better, the string "-lbcrypt" is needed in the gcc command 
invocation but is missing from the invocation. The actual library 
libbcrypt.a is _not_ missing. It's present in mingw's sys-root area, 
just as it should be, and is found automatically when the argument 
"-lbcrypt" is added to the gcc call.

Re: Cygwin targetting mingw 32-bit

[Vibhu M. <vi...@ho...>]

The Makefile's check-script target needs the following fix to run to 
completion.

System: Windows 10 64-bit
Cygwin (32-bit (and also likely 64-bit)) targetting mingw 32-bit

This is the complaint at the end of "make check-script"

rm -f script.lisp; echo '(eval-when (:compile-toplevel) (princ *args*))' 
 > script.lisp
output=`./lisp.exe -B . -E UTF-8 -Emisc 1:1 -Epathname 1:1 -norc -q -q 
-M lispinit.mem -c script.lisp foo bar`; test "$output" = '(foo bar)' || 
exit 1
make: *** [Makefile:2172: check-script] Error 1

The problem is that lisp.exe prints a DOS line ending (leaving $output 
with 11 bytes in total not 9).
The attached diff shows how to trim the carriage return when present. It 
does exactly what the Makefile already does over the next few lines for 
similar cases.

-- 
Vibhu

Re: Cygwin targetting mingw 32-bit

[Vibhu M. <vi...@ho...>]

To link the object files into an executable, -lbcrypt is needed but missing.

Processor: Intel Xeon X5647, x64-based processor
System: Windows 10 64-bit, 6 GB RAM
Cygwin (32-bit (and also likely 64-bit)) targetting mingw 32-bit

"make base" complains with:

i686-w64-mingw32-gcc -g -O2 -W -Wswitch -Wcomment -Wpointer-arith 
-Wreturn-type -Wmissing-declarations -Wimplicit -Wno-sign-compare 
-Wno-format-nonliteral -Wno-shift-negative-value -O2 
-fexpensive-optimizations -fwrapv -D_WIN32 -fno-strict-aliasing -DNO_ASM 
-DNO_ARI_ASM -DNO_SP_ASM -DNO_FAST_DISPATCH -DNO_FAST_FLOAT 
-DNO_FAST_DOUBLE -DNO_ALLOCA -DNO_ADDRESS_SPACE_ASSUMPTIONS 
-DNO_GENERATIONAL_GC -DNO_SYMBOLFLAGS -DENABLE_UNICODE 
-DNO_TERMCAP_NCURSES -DNO_GETTEXT -L/usr/local/mingw32/lib modules.o 
regexi.o calls.o -luser32 -lole32 -loleaut32 -luuid -lversion gettext.o 
lisp.a libgnu.a -luser32 -lole32 -loleaut32 -luuid -lws2_32 -o lisp.exe
/usr/lib/gcc/i686-w64-mingw32/11/../../../../i686-w64-mingw32/bin/ld: 
libgnu.a(getrandom.o): in function `getrandom':
/cygdrive/c/Users/vibhu/clisp_202209b/buildc32m6432a/gllib/../../src/gllib/getrandom.c:129: 
undefined reference to `BCryptGenRandom@16'
collect2: error: ld returned 1 exit status
./clisp-link: failed in 
/cygdrive/c/Users/vibhu/clisp_202209b/buildc32m6432a/base
make: *** [Makefile:2360: base] Error 1

I don't see -lbcrypt in there, which is the problem. The libs listed 
above with -l are all found as .a archives in mingw's sys-root area. As 
is libbcrypt.a . So adding -lbcrypt to the gcc command above fixes the 
problem so that "make base" succeeds.

I manually appended -lbcrypt to LIBS in my build directory's Makefile.
But the true fix is presumably upstream of that. I see that,
src/makemake.in
has a line adding -luser32 to LIBS. This should also add -lbcrypt.

Maybe it should also go into build/syscalls/link.sh's NEW_LIBS var.

-- 
Vibhu

Re: Cygwin targetting mingw 32-bit

[Vibhu M. <vi...@ho...>]

This is a fix to some configure files to deal with Windows line endings.

Processor: Intel Xeon X5647, x64-based processor
System: Windows 10 64-bit, 6 GB RAM
(I haven't yet gotten access to Windows 32-bit, but plan to)
Cygwin (both 64-bit and 32-bit) targetting mingw 32-bit

I've got lisp.exe and lispinit.mem and am seeing what it'll take to get 
to having a base/ directory.
I'll discuss the failing make targets i18n, syscalls.

Notice the extraneous carriage returns in lines 9, 11 of the attached 
extract of build/syscalls/config.log . They come from 
modules/syscalls/configure near "cl_cv_clisp_version=...". Sed's second 
expression does nothing because sed's stdin has a carriage return 
between the double quote and line feed.

Changing the expression to allow an optional carriage return like this:
's/"\r\?$//'
solves the problem.

There are some other similar places in that same configure script, which 
I also changed in exactly the same way. See attached diff. Moreover, I 
believe all the scripts named modules/*/configure will need the same 
lines fixed. If these scripts are generated, then perhaps the generator 
can change instead.

Note: This problem doesn't occur when building a Cygwin binary, only a 
mingw binary. That makes sense. The Cygwin binary presumably prints Unix 
line-endings, for which the original sed expression works just fine.

Re: Cygwin targetting mingw 32-bit

[Vibhu M. <vi...@ho...>]

On 27/09/2022 01:59, Bruno Haible wrote:
> Vibhu Mohindra wrote:
>> System: Windows 10 64-bit, 6 GB RAM
>> Processor: Intel Xeon X5647, x64-based processor
>> Cygwin (64-bit) targetting mingw 32-bit
>> clisp revision: f1aa42219433f019c05e707220538b3634bc233c
>> I've used the instructions in INSTALL.windows.
>>
>>
>> ----
>> 1. LN_S
>>
>> The Makefile has a variable LN_S that is chosen to be some platform
>> dependent value.
> 
> src/makemake.in says that valid values for this variable include
> 'ln -s', 'ln', and 'cp -p'.
> 
>> The reason is that build-aux is a directory, and hard linking
>> directories is not supported.
> 
> I don't think it's good for the Makefile to copy an entire build-aux
> directory. We should look into how to avoid that. Where is this
> happening?

Oh right. If LN_S is never meant to be applied to directories, then 
that's OK, and it's simply a matter of identifying where all it's used 
for directories.

The top level Makefile in the build directory (from one of my 
configurations) has this for the build-aux target.

build-aux : ../src/build-aux
	-$(RM) build-aux
	-$(LN_S) ../src/build-aux build-aux

A quick check with grep doesn't reveal any other places where LN_S is 
used for directories. Anyway, now that I know the intention, if I come 
across it being used anywhere else for directories, I'll fix that 
specific instance and write in, rather than trying to change the 
definition of LN_S itself.

If build-aux gets fixed in GitLab, I'll immediately pull that down.

>> Does "cp -a" seem a reasonable choice?
> 
> Yes, if directories need to be copied as well, then 'ln' and 'cp -p'
> won't work and 'cp -a' is better.
> 
>> 2. floatparam.h
>>
>> The generated floatparam.h has an error message at the bottom that looks
>> like it's causing make to fail.
> 
> Oh, this is something I last saw around 2000, when compilers were still
> buggy in some places. Often the error goes away if floatparam.c is compiled
> with less optimization options.
> 
>> So I re-ran the build afresh, but this
>> time immediately after the configure step, I replaced floatparam.h with
>> a copy from another build configuration I had lying around (Cygwin
>> targetting Cygwin 32-bit). Doing that got make further along.
>> Please see attached files: floatparam.h.orig (which is the actual one
>> generated for the mingw32 target build) and floatparam.h (the copy that
>> got me further, taken without good reason from a Cygwin build that
>> targetted Cygwin 32-bit).
> 
> Your floatparam.h is what I get as well on Linux/x86_64 with
> $ gcc -m32 -O2 floatparam.c && ./a.out && cat conftest.h
> So that is definitely expected (including the
>    #define double_rounds_correctly 0
> line).

Super to know, thanks a lot.

>> Background of what I'm up to:
>> (Please feel free to ignore all this)
>>
>> I'm building the various supported configurations listed in
>> INSTALL.windows. Some errors appear in many configurations. So I'll be
>> careful to try and list/resolve them in a sensible order and always be
>> clear which configuration I'm talking about. I presume nobody's tried
>> most of them in a long time and that they don't work for anybody any
>> more.
> 
> Thank you; yes, I haven't tried it on Cygwin since 2018 or so, and
> Cygwin as well as Windows evolves over time.

Oh, that's useful to know. It tells me my experiments needn't go farther 
back than 2018 GitLab. (Though it's now looking like going forward may 
work out, with all these useful notes you've given me.)

One thing to check. You've said elsewhere that 32-bit Cygwin binaries on 
64-bit Cygwin are problematic. Does that also apply to 32-bit mingw 
binaries on 64-bit Cygwin? If not, then I'll keep that configuration on 
my list instead of removing it. (That is, this thread's title stays in 
progress.) My initial goal is not to get unsupported or 
known-problematic configurations working. At first it's just to get all 
the known or thought-to-be-good configurations working for myself, in 
the spirit of walking before I can run.

>> Here's why I've embarked on this exercise in the first place. I've
>> written a command-line GPL'ed Lisp application for a non-technical
>> neighbour who only uses Windows. I don't know Windows well, but have now
>> got myself access to a Windows system. Besides pure clisp, the
>> application needs :ffi and thus ffcall. The most I can expect of my
>> neighbour is to double-click on an executable. So I'd like to be able to
>> give him a (GPL'ed) application rather than have him download clisp from
>> here, ffcall from there, then merge it all together. That would be
>> beyond him. So I plan to produce a single zip file with the application
>> binary and also all GPL-mandated sources, licences and build instructions.
> 
> Bravo! You have understood what the GPL is about. Even if your neighbour
> cannot rebuild things by himself, he could ask a friend or a contractor
> to do that for him.
> 
>> Note: I don't believe I'm allowed to distribute the official clisp mingw
>> binaries from sourceforge because its *features* have :ffi, so they have
>> ffcall in them. But they don't say which version. Or how to rebuild the
>> binaries (e.g. configure flags that were used). Even without ffcall,
>> I've been unable to produce mingw binaries from that old version of
>> clisp. So it seems against at least the spirit of the GPL for me to
>> distribute such binaries knowing that their instructions may have been
>> OK a decade ago on WinXP but don't work today and so their recipient
>> won't really be able to use them to rebuild the binary. In other words,
>> I shouldn't propagate a binary that I know the recipient can't modify
>> from source and rebuild because their instructions are obsolete.
> 
> Wow, you are giving more thought to the GPL than many other people!

Ha. The tiny application took 2%; writing a licence compliant build 
script and studying licences (18 Lisp libs, Quicklisp, clisp, mingw, 
Cygwin) 40%; building clisp on Windows 58% and counting :-)  It'll be 
fun to get it all perfect eventually. It's going to happen. I'm not in a 
hurry.

> Regarding the ffcall version: Yes, you are right, when distributing
> a binary that includes ffcall, the actual version should be specified.
> The excuse that we have for omitting this detail is:
>    - we have only used released versions, no snapshots, for building
>      public binaries,
>    - given that these releases maintained compatibility from 1.x to 1.y,
>      it does not really matter which release was used. The user will
>      get a working binary when rebuilding, regardless of which ffcall
>      version they choose.

Very helpful information, thanks. I believe therefore that if I bundle 
everything from sourceforge for clisp 2.49, removing the bz2 file 
(redundant) and mingw-big.zip file (so I don't need to think about the 
other libs in it), and then add the info in your note above about 
versions, that will be a bundling that respects all the various library 
authors' wishes.

Not that that's going to stop me from continuing to try producing my own 
build from the latest source. That still seems worthwhile. And is fun 
regardless.

> Regarding the configure flags: If they matter, they should be specified
> in the surrounding texts. If they don't matter, they can be omitted.
> The idea of the GPL is not to be able to reproduce exactly byte-for-byte the
> same binaries — that is a different goal, pursued by reproducible-builds.org —
> but merely to allow a user to apply their own modifications and get working
> binaries with these modifications.

Right, I meant configure flags only in the sense that that particular 
incantation was known to work and is worth getting working before trying 
others.

> Regarding the "instructions may have been OK a decade ago on WinXP but don't
> work today" argument: The GPL does not mandate source code that will
> compile fine 10 years in the future. Even the GCC 3.1, 3.3.6, 4.0.4 source
> does not compile out-of-the-box any more on current glibc systems. Thus,
> attempting a rebuild on Windows 10 of something that last worked on Windows XP
> is quite an adventure, yes. But the license cannot do anything about it.

Right, I understand that perfectly. It also means I don't need to 
provide instructions that will work perpetually. Even if they work for 
the moment on some computer, that's good enough. It's just that _I_ 
shouldn't propagate something that I can't build today (and therefore 
most hired programmers won't be able to build today either).

Thanks a lot for all your insight. I'm thrilled to hear from you and get 
this view into clisp.

Vibhu

Re: Cygwin 32-bit partial fix: memory map

[Vibhu M. <vi...@ho...>]

Hi Bruno,

On 27/09/2022 01:12, Bruno Haible wrote:
> Vibhu Mohindra wrote:
>> System: Windows 10, 6 GB RAM, Cygwin (64-bit)
>>
>> I'm now building 32-bit binaries as per INSTALL.windows.
>> ("3. Binaries for the Cygwin environment.")
> 
> Note that in my experience, 32-bit Cygwin binaries on a 64-bit Cygwin
> system don't work right regarding files and filesystem operations,
> depending on circumstances. If you have 64-bit Cygwin, I would thus
> recommend to build 64-bit Cygwin binaries. The INSTALL.windows said
> that libsigsegv is not supported on 64-bit Cygwin; however, since
> libsigsegv version 2.14 it is now supported there too.

Aha. Most useful piece of information. I misunderstood INSTALL.windows 
to think 64-bit Cygwin targetting both 32-bit Cygwin and 32-bit mingw 
was one of the supported configurations. That's the only reason I tried. 
The bottom of INSTALL.windows says to use Cygwin's setup-x86_64.exe, 
which is what misled me. But I realise x86_64 is just the architecture 
name and INSTALL.windows might well have meant I should use that Cygwin 
installer on a 32-bit computer/OS. Your note here probably explains a 
complaint I'm seeing from pathname.d in a different 32-bit target build. 
But to keep things tidy, I won't describe that in this thread.

>> The build produced this error:
>> Warning: reserving address range 0x80000000...0x8000ffff that contains
>> memory mappings. clisp might crash later!
>>
>> The attached change to lispbibl.c got past it. But it was guesswork from
>> scavenging this list, so I don't know if what I've done is definitely
>> right.
> 
> It is perfectly right; you have understood it well! :-)

Hurrah. Having the Makefile print memory layout and marc.out as well 
were brilliant ideas, whoever thought of them.

Super if this goes into GitLab, then I'll pull immediately and always 
reference that Git hash, rather than balancing a precarious stack of 
patches. (Fine if not too, as presumably there'll be a good reason and 
also I now know 64-bit targetting 32-bit is no longer an immediate goal.)

>> If I've understood correctly, one must pick a range not shown in
>> the log, so I did that. (The range I selected was the same as what a
>> neighbouring part of lispbibl.c had specified for win32_native/I80386.)
>>
>> The build then gets at least as far as producing lisp.exe and
>> lispinit.mem that runs in a sane looking way to me.
> 
> Cool! Btw, what's the Cygwin version you use? I use 2.9.0. Maybe the
> memory layout also depends on the Windows version; I don't know.

Oh I see. Right, inside a Cygwin terminal, uname -a prints this:
CYGWIN_NT-10.0-10240 mywin1 3.3.6-341.x86_64 2022-09-05 11:15 UTC x86_64 
Cygwin
Great, I can change to Cygwin 2.9.0 in due course if that turns out to 
matter.

>> Minor aside:
>>
>> I haven't thought about this deeply and may be wrong, but it may be
>> worth improving the Makefile in one way. I think if make fails with a
>> memory map complaint as above and you change no files but simply run
>> make again, it continues and produces a spurious lisp.exe . It would be
>> better for make to fail on its second invocation just as it did on its
>> first. I think the reason for it not failing on second invocation is
>> that marc.out has already been created, so it thinks that step
>> succeeded. One fix is to break marc.out creation into two steps: 1)
>> marc.out.tmp, and if check or whatever succeeded only then to 2) mv
>> marc.out.tmp marc.out
> 
> Hmm. It's not always clear what one wants. Sometimes the "spurious" lisp.exe
> will actually work. Sometimes it does not really work but the developer wants
> to use it under the debugger or strace. So, removing it would not be that
> good. Similarly for marc.out: "keep it simple", I would say.
> Generally what indicates a successful build is the presence of lisp.exe
> plus lispinit.mem.

Got it. So if my invocation of make ever takes more than one attempt to 
get all the way through (i.e. without wastefully cleaning each time), 
then I'll remember to only rely on the binaries if "make check" also 
says they are OK. Super, that'll be my strategy.

>> As a separate matter, the build doesn't get as far as making a base
>> directory. I'll post separately about that as it looks unrelated. So
>> far, I'm happy enough to have got a lisp.exe and lispinit.mem in the
>> build directory itself.
> 
> Yes, lisp.exe + lispinit.mem is the essential thing.

Brilliant. In that case, maybe I won't pursue this build configuration 
any further on a 64-bit machine. If INSTALL.windows also gets updated to 
say 64-bit targetting 32-bit is unsupported or has known file(system) 
problems, then others will also know.

Many thanks Bruno. I might take a bit of a break before resuming my 
overall building exercises. Got notes written carefully.

Vibhu

Re: Cygwin targetting mingw 32-bit

[Bruno H. <br...@cl...>]

Vibhu Mohindra wrote:
> System: Windows 10 64-bit, 6 GB RAM
> Processor: Intel Xeon X5647, x64-based processor
> Cygwin (64-bit) targetting mingw 32-bit
> clisp revision: f1aa42219433f019c05e707220538b3634bc233c
> I've used the instructions in INSTALL.windows.
> 
> 
> ----
> 1. LN_S
> 
> The Makefile has a variable LN_S that is chosen to be some platform 
> dependent value.

src/makemake.in says that valid values for this variable include
'ln -s', 'ln', and 'cp -p'.

> The reason is that build-aux is a directory, and hard linking 
> directories is not supported.

I don't think it's good for the Makefile to copy an entire build-aux
directory. We should look into how to avoid that. Where is this
happening?

> Does "cp -a" seem a reasonable choice?

Yes, if directories need to be copied as well, then 'ln' and 'cp -p'
won't work and 'cp -a' is better.

> 2. floatparam.h
> 
> The generated floatparam.h has an error message at the bottom that looks 
> like it's causing make to fail.

Oh, this is something I last saw around 2000, when compilers were still
buggy in some places. Often the error goes away if floatparam.c is compiled
with less optimization options.

> So I re-ran the build afresh, but this 
> time immediately after the configure step, I replaced floatparam.h with 
> a copy from another build configuration I had lying around (Cygwin 
> targetting Cygwin 32-bit). Doing that got make further along.
> Please see attached files: floatparam.h.orig (which is the actual one 
> generated for the mingw32 target build) and floatparam.h (the copy that 
> got me further, taken without good reason from a Cygwin build that 
> targetted Cygwin 32-bit).

Your floatparam.h is what I get as well on Linux/x86_64 with
$ gcc -m32 -O2 floatparam.c && ./a.out && cat conftest.h
So that is definitely expected (including the
  #define double_rounds_correctly 0
line).

> Background of what I'm up to:
> (Please feel free to ignore all this)
> 
> I'm building the various supported configurations listed in 
> INSTALL.windows. Some errors appear in many configurations. So I'll be 
> careful to try and list/resolve them in a sensible order and always be 
> clear which configuration I'm talking about. I presume nobody's tried 
> most of them in a long time and that they don't work for anybody any 
> more.

Thank you; yes, I haven't tried it on Cygwin since 2018 or so, and
Cygwin as well as Windows evolves over time.

> Here's why I've embarked on this exercise in the first place. I've 
> written a command-line GPL'ed Lisp application for a non-technical 
> neighbour who only uses Windows. I don't know Windows well, but have now 
> got myself access to a Windows system. Besides pure clisp, the 
> application needs :ffi and thus ffcall. The most I can expect of my 
> neighbour is to double-click on an executable. So I'd like to be able to 
> give him a (GPL'ed) application rather than have him download clisp from 
> here, ffcall from there, then merge it all together. That would be 
> beyond him. So I plan to produce a single zip file with the application 
> binary and also all GPL-mandated sources, licences and build instructions.

Bravo! You have understood what the GPL is about. Even if your neighbour
cannot rebuild things by himself, he could ask a friend or a contractor
to do that for him.

> Note: I don't believe I'm allowed to distribute the official clisp mingw 
> binaries from sourceforge because its *features* have :ffi, so they have 
> ffcall in them. But they don't say which version. Or how to rebuild the 
> binaries (e.g. configure flags that were used). Even without ffcall, 
> I've been unable to produce mingw binaries from that old version of 
> clisp. So it seems against at least the spirit of the GPL for me to 
> distribute such binaries knowing that their instructions may have been 
> OK a decade ago on WinXP but don't work today and so their recipient 
> won't really be able to use them to rebuild the binary. In other words, 
> I shouldn't propagate a binary that I know the recipient can't modify 
> from source and rebuild because their instructions are obsolete.

Wow, you are giving more thought to the GPL than many other people!

Regarding the ffcall version: Yes, you are right, when distributing
a binary that includes ffcall, the actual version should be specified.
The excuse that we have for omitting this detail is:
  - we have only used released versions, no snapshots, for building
    public binaries,
  - given that these releases maintained compatibility from 1.x to 1.y,
    it does not really matter which release was used. The user will
    get a working binary when rebuilding, regardless of which ffcall
    version they choose.

Regarding the configure flags: If they matter, they should be specified
in the surrounding texts. If they don't matter, they can be omitted.
The idea of the GPL is not to be able to reproduce exactly byte-for-byte the
same binaries — that is a different goal, pursued by reproducible-builds.org —
but merely to allow a user to apply their own modifications and get working
binaries with these modifications.

Regarding the "instructions may have been OK a decade ago on WinXP but don't
work today" argument: The GPL does not mandate source code that will
compile fine 10 years in the future. Even the GCC 3.1, 3.3.6, 4.0.4 source
does not compile out-of-the-box any more on current glibc systems. Thus,
attempting a rebuild on Windows 10 of something that last worked on Windows XP
is quite an adventure, yes. But the license cannot do anything about it.

Bruno

Re: Cygwin 32-bit partial fix: memory map

[Bruno H. <br...@cl...>]

Vibhu Mohindra wrote:
> System: Windows 10, 6 GB RAM, Cygwin (64-bit)
> 
> I'm now building 32-bit binaries as per INSTALL.windows.
> ("3. Binaries for the Cygwin environment.")

Note that in my experience, 32-bit Cygwin binaries on a 64-bit Cygwin
system don't work right regarding files and filesystem operations,
depending on circumstances. If you have 64-bit Cygwin, I would thus
recommend to build 64-bit Cygwin binaries. The INSTALL.windows said
that libsigsegv is not supported on 64-bit Cygwin; however, since
libsigsegv version 2.14 it is now supported there too.

> The build produced this error:
> Warning: reserving address range 0x80000000...0x8000ffff that contains 
> memory mappings. clisp might crash later!
> 
> The attached change to lispbibl.c got past it. But it was guesswork from 
> scavenging this list, so I don't know if what I've done is definitely 
> right.

It is perfectly right; you have understood it well! :-)

> If I've understood correctly, one must pick a range not shown in 
> the log, so I did that. (The range I selected was the same as what a 
> neighbouring part of lispbibl.c had specified for win32_native/I80386.)
> 
> The build then gets at least as far as producing lisp.exe and 
> lispinit.mem that runs in a sane looking way to me.

Cool! Btw, what's the Cygwin version you use? I use 2.9.0. Maybe the
memory layout also depends on the Windows version; I don't know.

> Minor aside:
> 
> I haven't thought about this deeply and may be wrong, but it may be 
> worth improving the Makefile in one way. I think if make fails with a 
> memory map complaint as above and you change no files but simply run 
> make again, it continues and produces a spurious lisp.exe . It would be 
> better for make to fail on its second invocation just as it did on its 
> first. I think the reason for it not failing on second invocation is 
> that marc.out has already been created, so it thinks that step 
> succeeded. One fix is to break marc.out creation into two steps: 1) 
> marc.out.tmp, and if check or whatever succeeded only then to 2) mv 
> marc.out.tmp marc.out

Hmm. It's not always clear what one wants. Sometimes the "spurious" lisp.exe
will actually work. Sometimes it does not really work but the developer wants
to use it under the debugger or strace. So, removing it would not be that
good. Similarly for marc.out: "keep it simple", I would say.
Generally what indicates a successful build is the presence of lisp.exe
plus lispinit.mem.

> As a separate matter, the build doesn't get as far as making a base 
> directory. I'll post separately about that as it looks unrelated. So 
> far, I'm happy enough to have got a lisp.exe and lispinit.mem in the 
> build directory itself.

Yes, lisp.exe + lispinit.mem is the essential thing.

Bruno

Cygwin targetting mingw 32-bit

[Vibhu M. <vi...@ho...>]

System: Windows 10 64-bit, 6 GB RAM
Processor: Intel Xeon X5647, x64-based processor
Cygwin (64-bit) targetting mingw 32-bit
clisp revision: f1aa42219433f019c05e707220538b3634bc233c
I've used the instructions in INSTALL.windows.


----
1. LN_S

The Makefile has a variable LN_S that is chosen to be some platform 
dependent value.

LN_S = ln
is not a suitable choice, at least for Cygwin targetting mingw 32-bit.
The reason is that build-aux is a directory, and hard linking 
directories is not supported.

LN_S = ln -s
might be a valid choice, but I don't know if symlinks would cause 
problems in the resulting native binary, or further in the build.

LN_S = cp -p
as recommended in some make/config file I can't remember offhand is also 
not a valid choice as that won't work for directories such as build-aux.

LN_S = cp -a
is probably a valid choice. I don't think the build ever does:
$(LN_S) x y
and then changes y expecting changes to be reflected in x.

Any tips on how I can reliably override LN_S in all primary and 
subsidiary make and config files will be most appreciated. My manual 
editing may not be addressing all required files and may be getting 
overwritten by some of the build. Does "cp -a" seem a reasonable choice?


----
2. floatparam.h

The generated floatparam.h has an error message at the bottom that looks 
like it's causing make to fail. So I re-ran the build afresh, but this 
time immediately after the configure step, I replaced floatparam.h with 
a copy from another build configuration I had lying around (Cygwin 
targetting Cygwin 32-bit). Doing that got make further along.
Please see attached files: floatparam.h.orig (which is the actual one 
generated for the mingw32 target build) and floatparam.h (the copy that 
got me further, taken without good reason from a Cygwin build that 
targetted Cygwin 32-bit).

I'd love to know what I need to do to have configure create a proper 
floatparam.h directly. (And also what a proper floatparam.h is to begin 
with, instead of my guessed copy.)

Note: I do always remember to start a fresh Cygwin terminal and to set 
PATH afresh and ulimit -s 16384 afresh. It's not the case that one build 
interferes with another.


----
Background of what I'm up to:
(Please feel free to ignore all this)

I'm building the various supported configurations listed in 
INSTALL.windows. Some errors appear in many configurations. So I'll be 
careful to try and list/resolve them in a sensible order and always be 
clear which configuration I'm talking about. I presume nobody's tried 
most of them in a long time and that they don't work for anybody any 
more. We know from the other recent thread that one configuration does 
work now, which is cool.

Of the various errors, the two above are good starts, being dependent on 
none of my other hacks trying to build things. They were produced by 
following INSTALL.windows to the letter.


Here's why I've embarked on this exercise in the first place. I've 
written a command-line GPL'ed Lisp application for a non-technical 
neighbour who only uses Windows. I don't know Windows well, but have now 
got myself access to a Windows system. Besides pure clisp, the 
application needs :ffi and thus ffcall. The most I can expect of my 
neighbour is to double-click on an executable. So I'd like to be able to 
give him a (GPL'ed) application rather than have him download clisp from 
here, ffcall from there, then merge it all together. That would be 
beyond him. So I plan to produce a single zip file with the application 
binary and also all GPL-mandated sources, licences and build instructions.

A mingw binary is valuable as it reduces how much I need to source, 
build and describe. I'll only need to source ffcall and clisp, then 
verify and write up how to rebuild them. A Cygwin binary on the other 
hand also requires me to source, build and document the production of 
cygwin1.dll (and cyggcc_s-1.dll). I'll do that work if I must, but nice 
to avoid. So a mingw target is valuable.

Note: I don't believe I'm allowed to distribute the official clisp mingw 
binaries from sourceforge because its *features* have :ffi, so they have 
ffcall in them. But they don't say which version. Or how to rebuild the 
binaries (e.g. configure flags that were used). Even without ffcall, 
I've been unable to produce mingw binaries from that old version of 
clisp. So it seems against at least the spirit of the GPL for me to 
distribute such binaries knowing that their instructions may have been 
OK a decade ago on WinXP but don't work today and so their recipient 
won't really be able to use them to rebuild the binary. In other words, 
I shouldn't propagate a binary that I know the recipient can't modify 
from source and rebuild because their instructions are obsolete.

Easily distributable clisps will make distributing GPL'ed clisp 
applications to non-technical people easy. That's what I'm after.


-- 
Vibhu

Cygwin 32-bit partial fix: memory map

[Vibhu M. <vi...@ho...>]

System: Windows 10, 6 GB RAM, Cygwin (64-bit)
clisp revision: f1aa42219433f019c05e707220538b3634bc233c

I'm now building 32-bit binaries as per INSTALL.windows.
("3. Binaries for the Cygwin environment.")

The build produced this error:
Warning: reserving address range 0x80000000...0x8000ffff that contains 
memory mappings. clisp might crash later!

The attached change to lispbibl.c got past it. But it was guesswork from 
scavenging this list, so I don't know if what I've done is definitely 
right. If I've understood correctly, one must pick a range not shown in 
the log, so I did that. (The range I selected was the same as what a 
neighbouring part of lispbibl.c had specified for win32_native/I80386.)

The build then gets at least as far as producing lisp.exe and 
lispinit.mem that runs in a sane looking way to me.

----
Minor aside:

I haven't thought about this deeply and may be wrong, but it may be 
worth improving the Makefile in one way. I think if make fails with a 
memory map complaint as above and you change no files but simply run 
make again, it continues and produces a spurious lisp.exe . It would be 
better for make to fail on its second invocation just as it did on its 
first. I think the reason for it not failing on second invocation is 
that marc.out has already been created, so it thinks that step 
succeeded. One fix is to break marc.out creation into two steps: 1) 
marc.out.tmp, and if check or whatever succeeded only then to 2) mv 
marc.out.tmp marc.out

----
As a separate matter, the build doesn't get as far as making a base 
directory. I'll post separately about that as it looks unrelated. So 
far, I'm happy enough to have got a lisp.exe and lispinit.mem in the 
build directory itself. Without the attached fix, I wouldn't get a sane 
lisp.exe/lispinit.mem .

Re: Cygwin build fix

[Bruno H. <br...@cl...>]

Vibhu Mohindra wrote:
> > Can you please try it out?
> 
> Tried your fix instead of mine. Worked perfectly.

Thank you for the quick return. I have now pushed the fix.

> [Separate matter:
> As before "make check" reports exactly one failure due to trying to open 
> /etc/passwd, which Cygwin doesn't have. I can't think of a file that's 
> definitely available on every platform, maybe /dev/zero or something or 
> even ./unix.c. This is tests/exceptsit.erg:
> Form: (PROGN (OPEN "/etc/passwd" :DIRECTION :INPUT :IF-EXISTS :ERROR) (/ 0))
> CORRECT: DIVISION-BY-ZERO
> CLISP  : #<SYSTEM::SIMPLE-FILE-ERROR #x00000008000A1D59>
> ]

Since you have checked that this file does not exist on your system, you can
obviously ignore the error. You're right that this file usually does not
exist any more on Cygwin: <https://stackoverflow.com/questions/28573763>.

Bruno

Re: Cygwin build fix

[Vibhu M. <vi...@ho...>]

Hi Bruno,

On 25/09/2022 04:41, Bruno Haible wrote:
> Vibhu Mohindra wrote:
>> My fix is to move the whole #define block a few lines down to after the
>> #include block. That prevents the #define macros from inadvertently
>> capturing symbols from the included files.
> 
> I think a less intrusive fix would be this one:
> 
> diff --git a/src/unix.d b/src/unix.d
> index c600b6cbb..f82b9f157 100644
> --- a/src/unix.d
> +++ b/src/unix.d
> @@ -566,7 +566,10 @@ extern int wait2 (pid_t pid); /* see unixaux.d */
>     #define WIN32_LEAN_AND_MEAN
>     #pragma push_macro ("Handle")
>     #undef Handle
> +  #pragma push_macro ("CR")
> +  #undef CR
>     #include <windows.h>
> +  #pragma pop_macro ("CR")
>     #pragma pop_macro ("Handle")
>     #undef WIN32
>     extern long time_t_from_filetime (const FILETIME * ptr);
> 
> Can you please try it out?

Tried your fix instead of mine. Worked perfectly.

[Separate matter:
As before "make check" reports exactly one failure due to trying to open 
/etc/passwd, which Cygwin doesn't have. I can't think of a file that's 
definitely available on every platform, maybe /dev/zero or something or 
even ./unix.c. This is tests/exceptsit.erg:
Form: (PROGN (OPEN "/etc/passwd" :DIRECTION :INPUT :IF-EXISTS :ERROR) (/ 0))
CORRECT: DIVISION-BY-ZERO
CLISP  : #<SYSTEM::SIMPLE-FILE-ERROR #x00000008000A1D59>
]

>> ----
>> I'm not an expert C programmer, but this technique of #including a file
>> after #defining a macro seems wrong in general. It makes changing the
>> included file impossibly hard as its author isn't free to use any new
>> symbols without fearing that an includer like clisp will capture them.
>> If clisp files followed a rule of never allowing #includes to follow
>> #defines, but to only ever precede them, I think that would consistently
>> prevent this particular sort of variable capture regardless of the depth
>> of an include hierarchy.
> 
> Such a rule is too rigid, and gets in the way of building larger software.
> Most importantly, for maintainable software, one should follow the principle
> that pieces of code that are semantically related should be close together.
> Putting the #includes in one place, the macros in a second place, the function
> declarations in a third place, and the function definitions in a fourth place
> tends to go against maintainability. (Early PASCAL was criticized for this:
> constants had to be defined first, then the types, then the functions, and
> finally the main program.)
> 
> It's better to not have this rule, and deal with the occasional (rare)
> conflicts on a case-by-case basis.

I see. So this isn't a new kind of problem. Everyone's obviously seen it 
before, and invented this macro pushing/popping. Cool stuff. Thanks for 
explaining.

>> An alternative prevention would be if include
>> file authors only created symbols with some namespace prefix, though
>> that would make their code unreadable.
> 
> Yes. Unfortunatelty, namespaces are a feature of newer programming languages,
> like C++ and Common Lisp itself, but not of C.

Oh right, I just meant namespaces the hard way: E.g. winnt.h would use 
_WINNT_CR rather than bare CR. But in any case, it would mean the whole 
world would have to agree on the convention before it could be useful, 
and it would look ugly anyway. Not as practical as the PASCAL or 
push/pop techniques.


Thanks a lot,

Vibhu

Re: Cygwin build fix

[Bruno H. <br...@cl...>]

Vibhu Mohindra wrote:
> I've found and fixed a problem in lispbibl.d. I've attached an 
> indicative diff of the C file.
> 
> System: Windows 10, a fresh Cygwin, and the latest GitLab clisp.
> 
> I followed the instructions in INSTALL.windows ("3. Binaries for the 
> Cygwin environment"), for 64-bit binaries. Besides the prescribed 
> configure options, I added --ignore-absence-of-libsigsegv.
> 
> Make fails to produce spvw.o because lispbibl.c does this near line 1900:
> #define CR 13
> and then #includes a file (unix.c) that ultimate #includes
> /usr/include/w32api/winnt.h
> which itself has a legitimate symbol (not macro) named CR that gets 
> replaced by 13 resulting in a syntax error.

Thanks for reporting this.

> My fix is to move the whole #define block a few lines down to after the 
> #include block. That prevents the #define macros from inadvertently 
> capturing symbols from the included files.

I think a less intrusive fix would be this one:

diff --git a/src/unix.d b/src/unix.d
index c600b6cbb..f82b9f157 100644
--- a/src/unix.d
+++ b/src/unix.d
@@ -566,7 +566,10 @@ extern int wait2 (pid_t pid); /* see unixaux.d */
   #define WIN32_LEAN_AND_MEAN
   #pragma push_macro ("Handle")
   #undef Handle
+  #pragma push_macro ("CR")
+  #undef CR
   #include <windows.h>
+  #pragma pop_macro ("CR")
   #pragma pop_macro ("Handle")
   #undef WIN32
   extern long time_t_from_filetime (const FILETIME * ptr);

Can you please try it out?

> ----
> I'm not an expert C programmer, but this technique of #including a file 
> after #defining a macro seems wrong in general. It makes changing the 
> included file impossibly hard as its author isn't free to use any new 
> symbols without fearing that an includer like clisp will capture them. 
> If clisp files followed a rule of never allowing #includes to follow 
> #defines, but to only ever precede them, I think that would consistently 
> prevent this particular sort of variable capture regardless of the depth 
> of an include hierarchy.

Such a rule is too rigid, and gets in the way of building larger software.
Most importantly, for maintainable software, one should follow the principle
that pieces of code that are semantically related should be close together.
Putting the #includes in one place, the macros in a second place, the function
declarations in a third place, and the function definitions in a fourth place
tends to go against maintainability. (Early PASCAL was criticized for this:
constants had to be defined first, then the types, then the functions, and
finally the main program.)

It's better to not have this rule, and deal with the occasional (rare)
conflicts on a case-by-case basis.

> An alternative prevention would be if include 
> file authors only created symbols with some namespace prefix, though 
> that would make their code unreadable.

Yes. Unfortunatelty, namespaces are a feature of newer programming languages,
like C++ and Common Lisp itself, but not of C.

Bruno