clisp-devel — closed list for developers and testers

Re: nightly build error fixed

[<don...@is...>]

Jean Louis writes:
 > On Sat, Dec 09, 2017 at 02:20:45PM +0000, Don Cohen wrote:
 > > 
 > > Don Cohen writes:
 > >  ...
 > >  >  > Makefile:41: recipe for target 'calls.o' failed
 > > 
 > > Just to confirm what whoever fixed this probably already knows,
 > > last night's build does not have the error reported yesterday.
 > 
 > I get now some new weird errors I did not have
 > before:

...

 > output="`./lisp.run -B . -N locale -E UTF-8 -Emisc 1:1 -Epathname 1:1 -norc -q -M lispinit.mem -x '(progn (setq *error-output* *standard-output*) (error "myerror"))' | tr -d '\r'`"; test "$output" = '*** - myerror' || exit 1
 > if ./lisp.run -B . -N locale -E UTF-8 -Emisc 1:1 -Epathname 1:1 -norc -q -M lispinit.mem -x '(progn (setq *error-output* *standard-output*) (error "myerror"))'; then exit 1; fi
 > *** - myerror
 > ./lisp.run -B . -N locale -E UTF-8 -Emisc 1:1 -Epathname 1:1 -norc -q -M lispinit.mem -x '(truename (make-stream :output))'
 > *** - TRUENAME: File #P"/dev/pts/3" does not exist
 > 
 > make: *** [Makefile:2237: check-script] Error 1
 > admin [ /sources/gnu/clisp-clisp/build-2017-12-09-15:28:18-Saturday ]$ ls /dev/pts/ptmx


I don't normally make check, but now that I try it, the form above does
not give me the problem you show above.  

I do see some other complaints:
 $  cat /home/don/hg/clisp/build-dir/tests/*.erg
 Form: (HANDLER-CASE (LETF ((*CURRENT-LANGUAGE* 'FRENCH)) (LIST (STRING= "À bientôt!" (SYSTEM::TEXT "Bye.")) (STRING= *CURRENT-LANGUAGE* "FRANÇAIS"))) (ERROR (E) (PRINC-ERROR E) '(T T)))
 CORRECT: (T T)
 CLISP  : (NIL T)
 Differ at position 0: T vs NIL
 CORRECT: (T T)
 CLISP  : (NIL T)


 Form: (WITH-OPEN-FILE (COPY S) (STREAMP COPY))
 CORRECT: T
 CLISP  : ERROR
 OS-FILE-ERROR(13): Permission denied
 OUT:
 "[OS-FILE-ERROR]: OS-FILE-ERROR(13): Permission denied
 "


 Form: (LET ((*REOPEN-OPEN-FILE* NIL)) (WITH-OPEN-FILE (COPY S :DIRECTION :OUTPUT) (STREAMP COPY)))
 CORRECT: T
 CLISP  : ERROR
 OS-FILE-ERROR(13): Permission denied
 OUT:
 "[OS-FILE-ERROR]: OS-FILE-ERROR(13): Permission denied
 "


I was not aware of either LETF or *CURRENT-LANGUAGE* before.
[1]> (LETF ((*CURRENT-LANGUAGE* 'FRENCH))  (SYSTEM::TEXT "Bye."))
"Bye."
I don't have a strong expectation of what the other two failed
test forms ought to do either.  

nightly build error fixed

[<don...@is...>]

Don Cohen writes:
 ...
 >  > Makefile:41: recipe for target 'calls.o' failed

Just to confirm what whoever fixed this probably already knows,
last night's build does not have the error reported yesterday.

Thanks.

Build problem on 64 bit Linux

[Blake M. <bl...@mc...>]

Hi,

I'm trying to build CLISP and running into the following problem:

$ hg pull
pulling from http://hg.code.sf.net/p/clisp/clisp
searching for changes
no changes found
$ ./configure
...
$ make distclean
cd src && if test -f Makefile; then make distclean; fi
rm -f Makefile
$ ./configure
...
configure: ** output file generation
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating po/Makefile.in
config.status: creating gllib/Makefile
config.status: creating makemake
config.status: creating ../Makefile
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing po-directories commands
config.status: creating po/POTFILES
config.status: creating po/Makefile
Configure findings:
  FFI:        yes (user requested: default)
  readline:   yes (user requested: default)
  libsigsegv: yes
$ cd src
$ make
make: *** No targets specified and no makefile found.  Stop.
blake@blake-sony ~/Backup/yy/clisp.hg/src
$ cd ..
$ make
cd src && make all
make[1]: Entering directory `/home/blake/Backup/yy/clisp.hg/src'
make[1]: *** No rule to make target `all'.  Stop.
make[1]: Leaving directory `/home/blake/Backup/yy/clisp.hg/src'
make: *** [all] Error 2
$


Appreciate any help.

Thanks!

Blake

nightly build errors

[<don...@is...>]

 > Fri Dec  8 03:07:27 PST 2017
 > pulling from http://hg.code.sf.net/p/clisp/clisp
 > searching for changes
 > adding changesets
 > adding manifests
 > adding file changes
 > added 4 changesets with 9 changes to 9 files
 > 9 files updated, 0 files merged, 0 files removed, 0 files unresolved

...
 > /home/don/hg/clisp/modules/syscalls/calls.c:2325:127: note: each undeclared identifier is reported only once for each function it appears in
 > /home/don/hg/clisp/modules/syscalls/calls.c: At top level:
 > /home/don/hg/clisp/modules/syscalls/calls.c:6750:6: warning: no previous declaration for  module__syscalls__init_function_1  [-Wmissing-declarations]
 > /home/don/hg/clisp/modules/syscalls/calls.c:6826:6: warning: no previous declaration for  module__syscalls__fini_function  [-Wmissing-declarations]
 > Makefile:41: recipe for target 'calls.o' failed
 > make[1]: *** [calls.o] Error 1
 > make[1]: Leaving directory '/home/don/hg/clisp/build-dir/syscalls'
 > Makefile:2332: recipe for target 'syscalls' failed
 > make: *** [syscalls] Error 2

I hope that's enough to tell the right people what they need to know.
If not, much more detail is available.

The output for the mt build is similar.

some ideas for multithreading support

[Bruno H. <br...@cl...>]

Hi all,

In 2008-2012, I couldn't give much input or feedback regarding the
multithreading implementation. But now, here are 3 ideas for improvement
that I collected over the last few months.

Vladimir: I appreciate a lot your hard work in this area. Especially the
GC and signal handling changes are among the most difficult things any hacker
can tackle.

  ----------------------------------------------------------------------
1) The philosophy of clisp and the philosophy of multithread support.

I find that it is a must that:

   Simultaneous access to the same object from different threads must,
   by default, lead to an error instead of a crash.

Why?

  * This is the basic philosophy of clisp: Simple programming mistakes
    lead to errors, not crashes. (And with decent error messages, in most
    cases.)

    While some implementations that compile to native code have default
    optimization settings that will allow a 'dotimes' loop, for example,
    to crash if the number is a bignum, clisp's philosophy implies that
    it does not optimize away type checks if that could lead to a crash.

    The point of keeping the type information on *all* objects, at runtime,
    is to be able to diagnose a type error in *all* situations. This is
    a distinguishing feature of clisp.

  * The philosophy of C, on the opposite side, is: Simple programming
    mistakes lead to crashes. This is the consequence of maximum optimization.
    When you program in C, you know that every time you make a small mistake,
    you risk a crash.

  * [1]: "If you want to shoot yourself, it is your responsibility to wear armor."
    This means that the philosophy of the multithreading support currently
    is the C philosophy.

  * I wish to have multithreading support with the clisp philosophy, not the
    C philosophy.

How to implement it?

*Every object access* will have to be protected by taking a lock.

For example,

  LISPFUNNR(car,1)
  {
    VALUES1(car(popSTACK()));
  }

will be rewritten to

  LISPFUNNR(car,1)
  {
    var object argument = popSTACK();
    RDLOCK(object);
    var object result = car(argument);
    RDUNLOCK(object);
    VALUES1(result);
  }

RDLOCK(object) may signal an error:
  "The object ~S is currently locked by thread ~S."

To this effect, *every object* will have, in its header, information
about which thread is currently holding a write-lock on the object, or
how many objects are holding a read-lock on the object. I believe this
can be done through a 32-bits field. (This field contains less information
than a "mutex" or "lock" in usual OS programming. Here RDLOCK will never
wait: it will either signal an error or proceed. It will never put the
current process in a "wait queue".)

There is literature that explains how to implement efficient locking,
e.g. "biased locking" [2].

  ----------------------------------------------------------------------
2) The set of target platforms.

At the lowest level, where clisp uses C functions from libc, a multithread
enabled clisp must only use multithread-safe function from libc. For
example, readdir_r instead of readdir. Except in cases where we can
guarantee that it's OK.

I don't know what the status on this task is, but I expect that this will
severely limit the set of target platforms for multithreading to glibc and
few other systems.

  ----------------------------------------------------------------------
3) Making use of standardized facilities

Since Vladimir wrote the multithreading support, standards have caught
up:
  * The '__thread' storage class is now also supported on Windows [3].
  * <stdatomic.h> (or std::atomic in C++) provides portable support for
    atomics.

I expect these facilities to be better optimized than what we could roll
on our own. Therefore it will be interesting to see to which extent we can
make use of them.

This too will limit the set of target platforms. In the end, I expect
that only modern glibc, Windows, and macOS will be left as target platforms
for multithreading.

  ----------------------------------------------------------------------

All this is food for the future. No hurry.

Please give yourself 24 hours of thinking before replying. [4]

Bruno

[1] https://clisp.sourceforge.io/impnotes/mt.html#mt-mutable
[2] https://www.cs.princeton.edu/picasso/mats/HotspotOverview.pdf
[2] https://msdn.microsoft.com/en-us/library/6yh4a9k1.aspx
[4] http://phk.freebsd.dk/sagas/bikeshed.html

Re: state of --with-threads=POSIX_THREADS?

[Vladimir T. <vtz...@gm...>]

I used to test threads builds on 32 bit OSX (even on PowerPC) but haven't
done so in the last year - currently I'm running and testing on 64 bit El
Capitan (10.11.2).
If there is general agreement that it's worthwhile I can spend time on 32
bit OSX support.

Also I'll try to handle timely bug reports for threading issues

--
  Vlad

On Thu, Dec 7, 2017 at 12:20 PM, <Joe...@t-...> wrote:

> Hi,
>
> What's the state of threads in the current clisp? Is it a release target?
> Is it worth opening bug reports about it?
>
> On my 32 bit OSX 10.6.8 machine, and in my environment (no ffcall, no
> sigsegv library!),
> all the regular build-porting-gcc-* expected to build (i.e. not the debug
> one, not the
> gen-gc ones) build fine and pass tests (except the flaky one, bug #720).
>
> However, trying out one of these configurations (the default one) with
> threads
> caused a core dump in threads.tst.
>
> I've not checked out whether one of the many other memory model
> configurations
> would work fine with threads.
>
> [In case you wonder, I'll add libsigsegv to my build environment at a
> later time. After all, I've
> repeated often enough in this list that users should not use clisp without
> it :-)]
>
> Regards,
>         Jörg
>

Re: multiple unsequenced modifications

[Sam S. <sd...@gn...>]

> * Bruno Haible <oe...@py...t> [2017-12-07 12:00:16 +0100]:
>
>> modules/rawsock/rawsock.c:893:48: warning: multiple unsequenced
>>   modifications to 'STACK' [-Wunsequenced]
>> It is worth bothering with?
>
> Definitely!

How is this different from -Wsequence-point ?

(cf https://sourceforge.net/p/clisp/patches/54/)

Thanks!

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net https://ffii.org
http://think-israel.org http://mideasttruth.com https://jihadwatch.org
To avoid fatigue, one must sleep 8 hours every day. And 8 hours every night too.

Re: multiple unsequenced modifications

[Sam S. <sd...@gn...>]

Hi Bruno,

> * Bruno Haible <oe...@py...t> [2017-12-07 12:00:16 +0100]:
>
>   memset(&hints,0,sizeof(hints));

Do we need to wrap this in begin_system_call()/end_system_call()?
I see a certain level of inconsistency about this. ;-(

Thanks!

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://no2bds.org
https://jihadwatch.org http://iris.org.il http://mideasttruth.com
To understand recursion, one has to understand recursion first.

Re: multiple unsequenced modifications

[Bruno H. <br...@cl...>]

Hi Sam,

> I get this:
> --8<---------------cut here---------------start------------->8---
> modules/rawsock/rawsock.c:893:48: warning: multiple unsequenced
>   modifications to 'STACK' [-Wunsequenced]
> --8<---------------cut here---------------end--------------->8---
> 
> because of
> 
> --8<---------------cut here---------------start------------->8---
>   struct addrinfo hints = {addrinfo_flags(),
>                            check_socket_domain(popSTACK()),
>                            check_socket_type(popSTACK()),
>                            get_socket_protocol(popSTACK()),
>                            0,NULL,NULL,NULL};
> --8<---------------cut here---------------end--------------->8---

GCC told you about one problem with this code.

The other problem is that the code assumes a certain order of
the members of 'struct addrinfo'. Which is not guaranteed by POSIX: [1] says
   "The <netdb.h> header shall define the addrinfo structure, which
    shall include at least the following members:"

> The only honest way to avoid the issue is
> 
> --8<---------------cut here---------------start------------->8---
>   struct addrinfo hints = {0,0,0,0,0,NULL,NULL,NULL};
>   hints.ai_flags = addrinfo_flags();
>   hints.ai_family = check_socket_domain(popSTACK());
>   hints.ai_socktype = check_socket_type(popSTACK());
>   hints.ai_protocol = get_socket_protocol(popSTACK());
> --8<---------------cut here---------------end--------------->8---

This is better, but it too suffers from the second problem.

How about this?

  struct addrinfo hints;
  memset(&hints,0,sizeof(hints));
  hints.ai_flags = addrinfo_flags();
  hints.ai_family = check_socket_domain(popSTACK());
  hints.ai_socktype = check_socket_type(popSTACK());
  hints.ai_protocol = get_socket_protocol(popSTACK());

> It is worth bothering with?

Definitely!

Bruno

[1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/netdb.h.html

Re: state of --with-threads=POSIX_THREADS?

[Bruno H. <br...@cl...>]

Hi Jörg,

> What's the state of threads in the current clisp? Is it a release target?

Not a release target for 2.50, nor for 2.51. Getting multithreading to
work reliably is a long-term target, that will take years to complete.

> Is it worth opening bug reports about it?

You can open such bugs; please mark them with 'Labels: multithreading'.

> On my 32 bit OSX 10.6.8 machine, and in my environment (no ffcall, no sigsegv library!),
> all the regular build-porting-gcc-* expected to build (i.e. not the debug one, not the
> gen-gc ones) build fine and pass tests (except the flaky one, bug #720).

Thanks a lot for this report!

Bruno

state of --with-threads=POSIX_THREADS?

[<Joe...@t-...>]

Hi,

What's the state of threads in the current clisp? Is it a release target?
Is it worth opening bug reports about it?

On my 32 bit OSX 10.6.8 machine, and in my environment (no ffcall, no sigsegv library!),
all the regular build-porting-gcc-* expected to build (i.e. not the debug one, not the
gen-gc ones) build fine and pass tests (except the flaky one, bug #720).

However, trying out one of these configurations (the default one) with threads
caused a core dump in threads.tst.

I've not checked out whether one of the many other memory model configurations
would work fine with threads.

[In case you wonder, I'll add libsigsegv to my build environment at a later time. After all, I've
repeated often enough in this list that users should not use clisp without it :-)]

Regards,
	Jörg

Re: Trouble with check-doc

[Karsten P. <Kar...@gm...>]

On 03.12.17 17:32, Sam Steingold wrote:

> 
> please try "hg pull -u" and then
> 
> make check-doc IMPNOTES=../doc/html/ CLHSROOT=http://clhs.lisp.se/
> 
> thanks.
> 
If I copy id-href.map manually to ../doc/html/id-href.map
make check-doc IMPNOTES=../doc/html/ CLHSROOT=http://clhs.lisp.se/
works fine, just in the Makefile I still have

IMPNOTES = "http://clisp.org/beta/impnotes/"
CLHSROOT = "http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/"
check-doc: clisp base
	IMPNOTES=$(IMPNOTES) ./clisp  -E UTF-8 -Emisc 1:1 -norc -x 
'(sys::ensure-impnotes-map t)'
	CLHSROOT=$(CLHSROOT) ./clisp  -E UTF-8 -Emisc 1:1 -norc -x 
'(sys::ensure-clhs-map)'

Karsten

multiple unsequenced modifications

[Sam S. <sd...@gn...>]

Hi,

I get this:
--8<---------------cut here---------------start------------->8---
modules/rawsock/rawsock.c:893:48: warning: multiple unsequenced
  modifications to 'STACK' [-Wunsequenced]
--8<---------------cut here---------------end--------------->8---

because of

--8<---------------cut here---------------start------------->8---
  struct addrinfo hints = {addrinfo_flags(),
                           check_socket_domain(popSTACK()),
                           check_socket_type(popSTACK()),
                           get_socket_protocol(popSTACK()),
                           0,NULL,NULL,NULL};
--8<---------------cut here---------------end--------------->8---

It appears that the code _should_ DTRT in all commercial compilers and
GCC 3.4.6 or later.

https://stackoverflow.com/q/19881803/850781
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=11633
http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#430

What should be done about this?

It is easy to rewrite the code (replace popSTACK() with STACK_??? and
add skipSTACK() at the end) but this would be a lie because
addrinfo_flags() modifies STACK too, so we are relying on the sequential
evaluation of all slots in the initialization no matter what.

The only honest way to avoid the issue is

--8<---------------cut here---------------start------------->8---
  struct addrinfo hints = {0,0,0,0,0,NULL,NULL,NULL};
  hints.ai_flags = addrinfo_flags();
  hints.ai_family = check_socket_domain(popSTACK());
  hints.ai_socktype = check_socket_type(popSTACK());
  hints.ai_protocol = get_socket_protocol(popSTACK());
--8<---------------cut here---------------end--------------->8---

Bruno, does it make sense?
It is worth bothering with?

Thanks

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://camera.org
http://americancensorship.org https://jihadwatch.org http://iris.org.il
"Bravery is being the only one who knows you're afraid." --Franklin P. Jones

Re: release plans and schedule

[Sam S. <sd...@gn...>]

Hi Jörg,

> *  <Wbret-Plevy.Ubruyr@g-flfgrzf.pbz> [2017-12-04 10:46:34 +0000]:
>
> 1. regexp module #691 
> ... --without-unicode ...

Unless you add a "without-unicode" target to the Makefile.devel
multibuild targets, your code will only be tested sporadically.
on the other hand, Bruno might already be running "make -f
Makefile.devel MULTIBUILD_EXTRAS=--without-unicode" ;-)

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net
http://islamexposedonline.com http://www.dhimmitude.org
Live Lisp and prosper.

Re: release plans and schedule

[Sam S. <sd...@gn...>]

Hi Jörg,

> *  <Wbret-Plevy.Ubruyr@g-flfgrzf.pbz> [2017-12-04 10:46:34 +0000]:
>
> 2. Hopefully loop.
> 2a. (loop for x = x or for x on x or for x from x) bugs #375 and #667
> I'm not convinced that Sam's recent change is TRT, but I need time to
> think about what clause goes where, read ANSI-CL and CLtL2 at least 7
> more times etc. and think about the meaning of the AND clause.

My recent change (52d209b7d29d) is certainly wrong, but I am not sure
how to fix it yet.
Please note that ANSI-CL and CLtL2 are _not_ the only thing to consider.
If all the other implementations consistently do something reasonable,
then this is the "user expectation" and it makes sense to comply (if
possible).

> Perhaps translate some of the remaining german comments, so Sam and
> others can benefit.

That's always welcome ;-)

> 2b. Maybe write a patch to signal an error from
> (loop for ... and for ...) ; no repetition of for/as after AND, as
> Pascal mentioned recently.

I am not sure this has high enough priority, but it would be nice, I
agree.

Thanks.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://camera.org
http://thereligionofpeace.com http://americancensorship.org http://jij.org
Those who don't know lisp are destined to reinvent it, poorly.

Re: release plans and schedule

[Sam S. <sd...@gn...>]

Hi Bruno,

> * Bruno Haible <oe...@py...t> [2017-12-03 18:49:02 +0100]:
>
> clisp now has much fewer build failures and random crashes that it had
> a year ago, and all of the "high-prority" items of my TODO list that I
> set up in February 2017 are fixed. Therefore my original plan would
> have been:
>
>   - fix remaining high-priority issues: bug 718.

I think 719 is also high-priority.
It is unconscionable that CLISP compiles its own libunistring and
libregexp on linux.
The current situation is that configure detects a good working
libunistring and sets LTLIBUNISTRING, but fails to massage it into LIBS.
However, I won't insist.

> Now it seems you (esp. Sam and Jörg) want to hack on something.

My TODO contains only LOOP.
If I cannot fix it on your schedule, you can roll back my recent changes
for the releases.
I think getting releases out should be the top priority.

> Therefore I guess we should at some point create a release branch, so
> that you can continue to hack with full speed and at the same time we
> limit the risk of breakage just before release. The precise question
> is WHEN to create this release branch.

I think we should revisit this when you are done with your
"pre-prerelease TODO" and ready to make the branch (in a week?)
At that point I will either say "LOOP is fixed" or "please create the
branch and roll back 52d209b7d29d".

Thanks!

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://jij.org
http://honestreporting.com http://no2bds.org http://camera.org
Suppose there is no empty set. Then consider the set of all empty sets.

Re: release plans and schedule

[<Joe...@t-...>]

Hi,

>What are your plans for the next few weeks? What would you like to hack on soon?
I have only 2 items:

1. regexp module #691
I need to change my patch. I've observed that almost no module works in a --without-unicode
setting, because most of them reference things like O(misc_encoding) or S(utf_8).
My patch allows to use #\ä inside both patterns and search strings.
- Patch a) Use O(misc_encoding), ignore --without-unicode.
  Nothing to debate here. Perhaps good enough already.
- Patch b) Change most misc_encoding to Symbol_value(utf_8),
  except the call to regerror which continues to use misc_encoding.
  Rationale: gnulib/regexp detects utf-8 internally -- if it has been compiled with Unicode.
  However, clisp cannot know how it was compiled.
- Patch c) Find some way to make the module (and perhaps other modules) work
  in a --without-unicode setting.


2. Hopefully loop.
2a. (loop for x = x or for x on x or for x from x) bugs #375 and #667
I'm not convinced that Sam's recent change is TRT, but I need time to think about
what clause goes where, read ANSI-CL and CLtL2 at least 7 more times etc. and
think about the meaning of the AND clause.

Perhaps translate some of the remaining german comments, so Sam and others can benefit.

My current reading is that there are significant differences in code motion, binding
and initialization between ANSI-CL and CLtL2 loop, but maybe I need to reread them and
hopefully find a way that can accommodate both without creating trouble.

For instance, CLtL2 clearly moves all initially clauses together inside one prog. CLISP alternates FOR and INITIALLY, IIRC. Or at least FOR and WITH.

There's an interesting example in CLtL2 where LOOP-FINISH is used inside INITIALLY
and references some of the initialized variables to cause termination.

2b. Maybe write a patch to signal an error from
(loop for ... and for ...) ; no repetition of for/as after AND, as Pascal mentioned recently.
CLtL2 did not allow that either.
I just read CLtL2 but did not look at that code part of loop.lsp

Regards,
	Jörg

release plans and schedule

[Bruno H. <br...@cl...>]

Hi all,

clisp now has much fewer build failures and random crashes that it had a year
ago, and all of the "high-prority" items of my TODO list that I set up in
February 2017 are fixed. Therefore my original plan would have been:

  - fix remaining high-priority issues: bug 718.
  - finish making assembly code portable.
  - Do a final round of testing on all Linux and GNU platforms.

  * PRERELEASE 2.49.70 -> for Debian

  - Do more rounds of testing on other platforms (from macOS to Haiku)

  * RELEASE 2.50 -> for the world

  - All other stuff comes afterwards, like https support, socket fixes,
    warning fixes etc.

Now it seems you (esp. Sam and Jörg) want to hack on something. Especially
during the end-of-December vacations, I guess.

But the timing of these releases is probably something like:

  * 2.49.70: between Dec 20 and Jan 7

  * 2.50: between Jan 25 and Feb 20

Therefore I guess we should at some point create a release branch, so that
you can continue to hack with full speed and at the same time we limit the risk
of breakage just before release. The precise question is WHEN to create
this release branch.

What are your plans for the next few weeks? What would you like to hack on
soon?

Bruno

Re: Trouble with check-doc

[Sam S. <sd...@gn...>]

Hi,

> * Karsten Poeck <Xnefgra.Cbrpx@tznvy.pbz> [2017-12-03 15:55:01 +0100]:
>
> check-doc: clisp base
> 	IMPNOTES="http://clisp.org/beta/impnotes/" ./clisp  -E UTF-8
> -Emisc 1:1 -norc -x '(sys::ensure-impnotes-map t)'
> 	#IMPNOTES="https://clisp.sourceforge.io/impnotes/" ./clisp  -E
> UTF-8 
> -Emisc 1:1 -norc -x '(sys::ensure-impnotes-map t)'
> 	CLHSROOT="http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec"
> ./clisp  -E UTF-8 -Emisc 1:1 -norc -x '(sys::ensure-clhs-map)'

please try "hg pull -u" and then

make check-doc IMPNOTES=../doc/html/ CLHSROOT=http://clhs.lisp.se/

thanks.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://www.dhimmitude.org
http://mideasttruth.com http://no2bds.org http://think-israel.org
UNIX, car: hard to learn/easy to use; Windows, bike: hard to learn/hard to use.

Trouble with check-doc

[Karsten P. <Kar...@gm...>]

Hello,
recently (for sure with todays hg tip) I seem to have trouble with 
check-doc from the check step in the makefile.

IMPNOTES seem to be generated from makemake.in, ignoring what I put in 
cfgunix.lsp

..
  MYIMPROOT=http://clisp.org/beta/impnotes/
  MYTESTURL=${MYIMPROOT}id-href.map
..

http://clisp.org/beta/impnotes/ forwards to 
https://clisp.sourceforge.io/beta/impnotes/ but ext:open-url does not 
handle https: (as perhaps discussed earlier)

How should I configure this correctly?

I saw a change that disables the internet access in streams.tst, but 
that does not seem to help here.

regards

Karsten

 From Makefile
check-doc: clisp base
	IMPNOTES="http://clisp.org/beta/impnotes/" ./clisp  -E UTF-8 -Emisc 1:1 
-norc -x '(sys::ensure-impnotes-map t)'
	#IMPNOTES="https://clisp.sourceforge.io/impnotes/" ./clisp  -E UTF-8 
-Emisc 1:1 -norc -x '(sys::ensure-impnotes-map t)'
	CLHSROOT="http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec" 
./clisp  -E UTF-8 -Emisc 1:1 -norc -x '(sys::ensure-clhs-map)'

Re: makemake.in configuration (os--hw) -- what default settings?

[<Joe...@t-...>]

Bruno wrote:

>Once you have determined this set, remove most optimization blocker flags
>in makemake.in and redo
>$ make -k -f Makefile.devel multibuild-macosx-i386
>and verify that all builds that succeeded earlier still succeed.

Good news. On that 32bit 2006 Macbook Pro I could remove all safeties
NO_*ASM, NO_FAST_*, set SAFETY=0
and there were no new findings. Gcc was Apple's (Xcode DVD or combo updates).

I'm now testing some other configurations, e.g. --without-unicode, or OLD_GC.

Regards,
	Jörg

Re: optional argument in foreign calls

[Sam S. <sd...@gn...>]

Thanks Jörg!
Looks like your approach is what is already being done in pari.lisp
(see pari-call-out)

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net https://jihadwatch.org
http://americancensorship.org http://thereligionofpeace.com
"Bravery is being the only one who knows you're afraid." --Franklin P. Jones

Re: optional argument in foreign calls

[<Joe...@t-...>]

Sam,

https://pari.math.u-bordeaux.fr/dochtml/html/Arithmetic_functions.html

"The library syntax is GEN gp_factor0(GEN x, GEN lim = NULL).
This function should only be used by the gp interface.
Use directly GEN factor(GEN x) or GEN boundfact(GEN x, ulong lim)."

I don't know what the "gp interface" is. Yet to me it seems like there are 2 low-level
functions, factor/1 and boundfact/2. Perhaps gp is a high-level shell that gives
the user the illusion of a single function with optional parameters.

Regards,
	Jörg

Re: optional argument in foreign calls

[<Joe...@t-...>]

Hi,

Sam Steingold wrote:
--8<---------------cut here---------------start------------->8---
factor(x,{lim}): factorization of x. lim is optional and can be set whenever x is of (possibly recursive) rational type. If lim is set return partial factorization, using primes < lim.
--8<---------------cut here---------------end--------------->8---
>-- do we really need *two* def-call-out forms?)

[Please provide a link to the PARI/GP doc of that function, because variadic stuff in C
 is always hairy.]

Short: Yes. The FFI is an automata that (de)serializes values based on an interface
description that it parses. There's no provision (yet) for optional parameters.
That would be too much ad hoc, as the following examples show.

I'd provide the long form only, use &optional at the level of a Lisp wrapper function,
and always supply the optional parameter, possibly as a dummy that the callee C function
can ignore. Basically, *ignoring* extra parameters on the stack is what variadic
functions do, with their decision based on analyzing the first (and required) parameters.

E.g.
- The printf family walks the stack according to a string like "%s%lld%c".
  Kaboom if the format string doesn't match the actual parameters (and their sizes).
  That's why modern C compilers insist on checking that string.
- open(pathname, flags[, mode]) works like this:
             "This [mode] argument must be supplied when
              O_CREAT or O_TMPFILE is specified in flags; if neither
              O_CREAT nor O_TMPFILE is specified, then mode is ignored."
This works, because in the C calling convention, the callee has no idea how many arguments
were supplied but it sees the first arguments on the top of stack (unlike Pascal).

IOW, call open() with 0 (or ~0) as 3rd argument when not supplied and everything
shall be fine, because it ought to have been supplied whenever it is necessary.

Your snippet from factor() is not enough for me to understand when lim is optional.
It somehow must depend on properties of x. But it can never be optional in the sense
of Prolog, Erlang, Ada or C++, with 2 distinct interfaces. In C, a unique function
exists under a given name and must have means to know whether to expect and
retrieve 2 or 3 arguments from the stack.

"It can be set" is no good. "It *must* be set when ..." is needed with C.

Regards,
	Jörg

Re: when do we remove deprecated features?

[Bruno H. <br...@cl...>]

Hi Sam,

> 1. We should keep the CLtL2 and ANSI CL deprecated symbols.

I agree. There are some old packages that are still useful but don't
have a maintainer. It would be a disgrace to break them just because of
past evolutions of the standard.

> 2. We can drop our extensions:
> 
> type-expand-1:        deprecated since 2002-01-08
> foreign-address-null: deprecated since 2003-08-05
> rename-dir, make-dir, delete-dir:      2007-12-31
> socket-service-port:  deprecated since 2005-11-09

A reasonable rule would be: We can remove an extension 5 years after the first
release in which it became deprecated.

All of the above can be removed according to this rule.

> 3. We should keep wildcard (deprecated since 2011-05-17, after the last release)

Yes, agree.

Bruno