clisp-devel

Trouble building 2.49.60 beta on Fedora

[Jerry J. <log...@gm...>]

Greetings,

I updated the Fedora ffcall package to the 1.13 release awhile ago.
Recently I attempted to update clisp to the 2.49.60 beta release as
well, but I'm having some trouble with it.  The x86 builds, both
32-bit and 64-bit, are fine, as is the 32-bit ARM build.  However, the
s390x, ppc64 and ppc64le builds all fail with this message:

../src/lispbibl.d:2987:2: error: #error CLISP has not been ported to
this platform - oint_type_len undefined
 #error CLISP has not been ported to this platform - oint_type_len undefined

Are additional configure flags of some kind needed when building for
those platforms?

Also, the aarch64 build fails like this:

;; Compiling file /builddir/build/BUILD/clisp-2.49.60/modules/asdf/asdf.lisp ...
*** - handle_fault error2 ! address = 0x2e0800084480 not in
[0x2e0000000000,0x2e0000066778) !
SIGSEGV cannot be cured. Fault address = 0x2e0800084480.
GC count: 14
Space collected by GC: 12036056
Run time: 0 457240
Real time: 0 476000
GC time: 0 38463
Permanently allocated: 165368 bytes.
Currently in use: 4392320 bytes.
Free space: 299880 bytes.
make[1]: Leaving directory '/builddir/build/BUILD/clisp-2.49.60/build/asdf'
make: Leaving directory '/builddir/build/BUILD/clisp-2.49.60/build'
make[1]: *** [Makefile:22: asdf.fas] Segmentation fault (core dumped)
make[1]: *** Deleting file 'asdf.fas'
make: *** [Makefile:2334: asdf] Error 2

Do you have any idea what might cause that?  Finally, I needed the
attached patch to build successfully with gdbm 1.13.

Regards,
-- 
Jerry James
http://www.jamezone.org/

Re: Trouble building 2.49.60 beta on Fedora - gdbm

[Bruno H. <br...@cl...>]

Hello Jerry,

Thanks for the detailed reports!

I'll reply for each of the issues separately.

> Finally, I needed the
> attached patch to build successfully with gdbm 1.13.

Thanks for the report. I'll use this patch (in order to continue supporting
gdbm 1.8.3).

Bruno


diff -r 5b5ef50761bd modules/gdbm/gdbm.c
--- a/modules/gdbm/gdbm.c	Sun Jun 25 11:46:27 2017 +0200
+++ b/modules/gdbm/gdbm.c	Sun Aug 20 07:08:22 2017 +0200
@@ -66,7 +66,7 @@
            READER-CANT-REORGANIZE UNKNOWN-UPDATE ITEM-NOT-FOUND              \
            REORGANIZE-FAILED CANNOT-REPLACE ILLEGAL-DATA OPT-ALREADY-SET     \
            OPT-ILLEGAL)
-static _Noreturn void error_gdbm (char *fatal_message) {
+static _Noreturn void error_gdbm (const char *fatal_message) {
   end_blocking_system_call(); /* in case we are called from _gdbm_fatal() */
   pushSTACK(`GDBM::GDBM-ERROR`);
   pushSTACK(`:MESSAGE`);
@@ -124,10 +124,16 @@
 
 static object open_gdbm (object path, int bsize, int rw, int mode) {
   GDBM_FILE gdbm;
+  /* gdbm versions < 1.9 had a broken prototype of gdbm_open() in gdbm.h. */
+  #if !defined(GDBM_OPENMASK) && defined(__cplusplus)
   with_string_0(path, GLO(pathname_encoding), name, {
-      SYSCALL(gdbm = gdbm_open(name, bsize, rw, mode,
-                               (void (*)(void))error_gdbm));
-    });
+    SYSCALL(gdbm = gdbm_open(name, bsize, rw, mode, (void (*) (void)) error_gdbm));
+  });
+  #else
+  with_string_0(path, GLO(pathname_encoding), name, {
+    SYSCALL(gdbm = gdbm_open(name, bsize, rw, mode, error_gdbm));
+  });
+  #endif
   if (gdbm == NULL) error_gdbm(NULL);
   return allocate_fpointer(gdbm);
 }

Re: Trouble building 2.49.60 beta on Fedora - gdbm

[Jerry J. <log...@gm...>]

Hi Bruno,

On Sat, Aug 19, 2017 at 11:22 PM, Bruno Haible <br...@cl...> wrote:
> Thanks for the report. I'll use this patch (in order to continue supporting
> gdbm 1.8.3).

That looks great, but I don't see anything accounting for the
UNKNOWN-UPDATE to UNKNOWN-ERROR name change.  I'm not sure which
version of gdbm made that change, but I can look into it if you need
me to do so.

Regards,
-- 
Jerry James
http://www.jamezone.org/

Re: Trouble building 2.49.60 beta on Fedora - arm64

[Bruno H. <br...@cl...>]

Hi Jerry,

> Also, the aarch64 build fails like this:
> 
> ;; Compiling file /builddir/build/BUILD/clisp-2.49.60/modules/asdf/asdf.lisp ...
> *** - handle_fault error2 ! address = 0x2e0800084480 not in
> [0x2e0000000000,0x2e0000066778) !
> SIGSEGV cannot be cured. Fault address = 0x2e0800084480.
> GC count: 14
> Space collected by GC: 12036056
> Run time: 0 457240
> Real time: 0 476000
> GC time: 0 38463
> Permanently allocated: 165368 bytes.
> Currently in use: 4392320 bytes.
> Free space: 299880 bytes.
> make[1]: Leaving directory '/builddir/build/BUILD/clisp-2.49.60/build/asdf'
> make: Leaving directory '/builddir/build/BUILD/clisp-2.49.60/build'
> make[1]: *** [Makefile:22: asdf.fas] Segmentation fault (core dumped)
> make[1]: *** Deleting file 'asdf.fas'
> make: *** [Makefile:2334: asdf] Error 2
> 
> Do you have any idea what might cause that?

For arm64 support, please use the attached patch.

Bruno

Re: Trouble building 2.49.60 beta on Fedora - arm64

[Bruno H. <br...@cl...>]

> For arm64 support, please use the attached patch.

Oops, small mistake. Please use this one instead.

Bruno

Re: Trouble building 2.49.60 beta on Fedora - arm64

[Jerry J. <log...@gm...>]

On Sun, Aug 20, 2017 at 3:55 PM, Bruno Haible <br...@cl...> wrote:
>> For arm64 support, please use the attached patch.
>
> Oops, small mistake. Please use this one instead.

Again, thank you for the quick response.  I will try a build with
these patches in the next 24 hours and let you know how it goes.
Regards,
-- 
Jerry James
http://www.jamezone.org/

Re: Trouble building 2.49.60 beta on Fedora - powerpc64

[Bruno H. <br...@cl...>]

Jerry James wrote:
> the s390x, ppc64 and ppc64le builds all fail with this message:
> 
> ../src/lispbibl.d:2987:2: error: #error CLISP has not been ported to
> this platform - oint_type_len undefined
>  #error CLISP has not been ported to this platform - oint_type_len undefined

For powerpc64 and powerpc64le, please use the attached patch.

Bruno

Re: Trouble building 2.49.60 beta on Fedora - powerpc64

[Jerry J. <log...@gm...>]

On Sun, Aug 20, 2017 at 4:02 PM, Bruno Haible <br...@cl...> wrote:
> For powerpc64 and powerpc64le, please use the attached patch.

Wonderful!  Thank you so much for the quick response.  Is s390x
unsupported for now?  If so, I can exclude it from making clisp
builds.

I will give this patch a try soon, probably tomorrow night.

Regards,
-- 
Jerry James
http://www.jamezone.org/

Re: Trouble building 2.49.60 beta on Fedora - s390x

[Bruno H. <br...@cl...>]

Jerry James wrote:
> Is s390x unsupported for now?

I am testing a patch for s390x support. But my build machine is quite slow [1],
therefore this testing takes 24 hours.

Bruno

[1] http://git.savannah.gnu.org/gitweb/?p=libffcall.git;a=blob_plain;f=porting-tools/emulation/qemu-s390x.txt

Re: Trouble building 2.49.60 beta on Fedora - s390x

[Jerry J. <log...@gm...>]

On Mon, Aug 21, 2017 at 12:24 AM, Bruno Haible <br...@cl...> wrote:
> I am testing a patch for s390x support. But my build machine is quite slow [1],
> therefore this testing takes 24 hours.

Yes, QEMU emulation is quite slow.  I did not mean to sound impatient.
Anyway, I have encountered some technical difficulties tonight,
unrelated to clisp, which have prevented me from making progress.  I
will try again tomorrow.

Regards,
-- 
Jerry James
http://www.jamezone.org/

Re: Trouble building 2.49.60 beta on Fedora

[Jerry J. <log...@gm...>]

Thanks again, Bruno, for the patches.  They don't apply cleanly to
either the 2.49.60 beta release or to current mercurial head, though.
What version of the code are those patches against?

Sadly, the patches have not improved the situation.  The 32- and
64-bit x86 and 32-bit ARM builds still succeed, and all of the others
segfault now.  I'm not sure if you can follow this link or not, but if
you can, build logs will be here for a few days:

https://koji.fedoraproject.org/koji/taskinfo?taskID=21412890

That build was against mercurial revision 820ad4b724cb, with the
following patches:

- A patch to fix the ARM build:
http://sourceforge.net/tracker/?func=detail&aid=3529615&group_id=1355&atid=301355
- A patch to fix configure detection of Berkeley DB:
http://sourceforge.net/tracker/?func=detail&aid=3572516&group_id=1355&atid=301355
- A patch to add definitions for ERFKILL and EHWPOISON, and to remove
the definition of cfree, which has been dropped from recent versions
of glibc (attached).
- The gdbm patch I sent with the initial email
- The ppc64 and aarch64 patches you have sent in this thread, adapted
to apply to the mercurial revision noted above.

Also, Fedora builds with -Werror=format-security, so I had to change
one line in the aarch64 patch.  That patch modifies spvw_allocate.d
and in one place issues a warning.  I had to change this line:

fprintf(stderr, GETTEXTL("Warning: "));

to:

fputs(GETTEXTL("Warning: "),stderr);

since the string is non-constant, and therefore triggers the format
security error.

I just noticed that we add these flags when building for ppc64/ppc64le:

-DNO_GENERATIONAL_GC -DNO_MULTIMAP_FILE -DNO_SINGLEMAP

That part of the spec file predates my involvement, so I don't know
why they are in there.  Should we be using those flags?

Please let me know if there is anything I can do to help debug these
segfaults further.  Regards,
-- 
Jerry James
http://www.jamezone.org/

Re: Trouble building 2.49.60 beta on Fedora

[Bruno H. <br...@cl...>]

Hi Jerry,

> Thanks again, Bruno, for the patches.  They don't apply cleanly to
> either the 2.49.60 beta release or to current mercurial head, though.
> What version of the code are those patches against?

The patches were meant to apply to the 2.49.60 beta release. They do
apply without rejects, if you apply them in the order I sent them:

$ cd clisp-2.49.60/
$ patch -p1 < ../gdbm.diff
patching file modules/gdbm/gdbm.c
$ patch -p1 < ../arm64.diff 
patching file src/lispbibl.d
Hunk #1 succeeded at 390 (offset -4 lines).
Hunk #2 succeeded at 2898 (offset -4 lines).
Hunk #3 succeeded at 3029 (offset -4 lines).
Hunk #4 succeeded at 3048 (offset -4 lines).
Hunk #5 succeeded at 3068 (offset -4 lines).
Hunk #6 succeeded at 3966 (offset -4 lines).
patching file src/makemake.in
patching file src/spvw.d
patching file src/spvw_allocate.d
$ patch -p1 < ../powerpc64.diff 
patching file src/lispbibl.d
Hunk #1 succeeded at 2917 (offset -4 lines).
Hunk #2 succeeded at 3406 (offset -4 lines).
patching file src/spvw.d
$ patch -p1 < ../s390x.diff 
patching file src/lispbibl.d
Hunk #1 succeeded at 2378 (offset -4 lines).
Hunk #2 succeeded at 2388 (offset -4 lines).
Hunk #3 succeeded at 2420 (offset -4 lines).
Hunk #4 succeeded at 2621 (offset -4 lines).
Hunk #5 succeeded at 2942 (offset -4 lines).
patching file src/spvw.d

> Sadly, the patches have not improved the situation.  The 32- and
> 64-bit x86 and 32-bit ARM builds still succeed, and all of the others
> segfault now.
> That build was against mercurial revision 820ad4b724cb, with the
> following patches: ...

Please use the 2.49.60 beta release as a starting point. If you use
the mercurial HEAD, you live more dangerously. There are some kinds of
bugs that we detect and fix using special testing modes during beta testing,
and this has not yet happened for the changes between 2.49.60 and HEAD.

> - A patch to fix the ARM build:
> http://sourceforge.net/tracker/?func=detail&aid=3529615&group_id=1355&atid=301355
> - A patch to fix configure detection of Berkeley DB:
> http://sourceforge.net/tracker/?func=detail&aid=3572516&group_id=1355&atid=301355
> - A patch to add definitions for ERFKILL and EHWPOISON, and to remove
> the definition of cfree, which has been dropped from recent versions
> of glibc (attached).
> - The gdbm patch I sent with the initial email

These patches look all reasonable. They are surely not the causes of crashes
on Linux/x86_64.

If you still get segfaults on all other platforms, it may be due to
  * optimization bugs in the GCC you are using: please patch makemake.in
    to use only -O instead of -O2 and try that,
  * different behaviour of the Linux kernel. There is a crude workaround,
    which is to configure with --enable-portability, but this disables
    a lot of optimization features (including generational GC), therefore
    this is not the first thing I'd recommend to you. The other approach
    would be for me to debug it in a VM for x86_64 or i386. Which Fedora
    installer image is it that I can download and try in a VM?

> I just noticed that we add these flags when building for ppc64/ppc64le:
> 
> -DNO_GENERATIONAL_GC -DNO_MULTIMAP_FILE -DNO_SINGLEMAP
> 
> That part of the spec file predates my involvement, so I don't know
> why they are in there.  Should we be using those flags?

I tested the powerpc64.diff without these options, therefore I'm pretty
sure you can remove them.

Bruno

Re: Trouble building 2.49.60 beta on Fedora

[Bruno H. <br...@cl...>]

Hi Jerry,

> Fedora builds with -Werror=format-security, so I had to change
> one line in the aarch64 patch.  That patch modifies spvw_allocate.d
> and in one place issues a warning.  I had to change this line:
> 
> fprintf(stderr, GETTEXTL("Warning: "));
> 
> to:
> 
> fputs(GETTEXTL("Warning: "),stderr);
> 
> since the string is non-constant, and therefore triggers the format
> security error.

Thanks for the heads-up. We have now added two workarounds; each of them
would be sufficient to avoid this warning.

Bruno

Re: Trouble building 2.49.60 beta on Fedora

[Bruno H. <br...@cl...>]

Jerry James wrote:
> the s390x, ppc64 and ppc64le builds all fail with this message:
> 
> ../src/lispbibl.d:2987:2: error: #error CLISP has not been ported to
> this platform - oint_type_len undefined
>  #error CLISP has not been ported to this platform - oint_type_len undefined

For s390x support, please try this patch.

Bruno