clisp-devel — closed list for developers and testers

Re: lazy-let & loop

[<Joe...@t-...>]

Sam,

I think I've solved the problem of initialization of LOOP forms -- in my head. I need time to write it down
and show you the results. No need for lazy-let.

I've not yet attacked the problem of iteration in LOOP. I'm afraid there will be no way to
make everybody happen w.r.t. the value of variables in FINALLY clauses.

But at least I believe we can make everyone happy w.r.t. the semantics of clauses such as FOR vars ON vars,
in all tricky combinations (ordering of clauses, joining with AND).

Regards,
	Jörg

lazy-let & loop

[Sam S. <sd...@gn...>]

Hi Bruno, Jörg,

The following is an elaboration on
https://sourceforge.net/p/clisp/bugs/667/#2b06

The problem with loop is that under certain circumstances we separate
initializing and binding of a variable.

I wonder if something like "lazy-binding" could help: bind the variable
but not initialize its value. until it is explicitly set, accessing it
reaches up in the environment. This would require something like
`lazy-let` with the following test case:

--8<---------------cut here---------------start------------->8---
(let ((a 1))
  (lazy-let (a b)
    (print a) ; prints 1
    (print b) ; error - unbound variable
    (setq a 10) ; inner binding of a is activated
    (print a) ; prints 10
    (setq b 22)) ; binding of b is activated
  (print b) ; error - unbound variable
  (print a)) ; prints 1
--8<---------------cut here---------------end--------------->8---

while it is easy to avoid `lazy-let` above by putting `let` after the
second `print`, it is not so easy with `loop` because we need the
binding to exist around the main `tagbody` but to do initialization in
the preamble inside `tagbody`.

Loop expansion then will loop like this:

--8<---------------cut here---------------start------------->8---
(lazy-let (all the loop local variables thay are now bound by let)
  (tagbody 
     preamble - set all the loop local variables thay are now bound by let
     ...
     )) ; close tagbody and lazy-let
--8<---------------cut here---------------end--------------->8---

BTW, we can merge `lazy-let` and `tagbody` into a single form `taglet`:

--8<---------------cut here---------------start------------->8---
(taglet (variables to lazily bind)
  forms...
  tags...)
--8<---------------cut here---------------end--------------->8---

Alternatively, `lazy-let` can be implemented as something like

--8<---------------cut here---------------start------------->8---
(let ((a (if (boundp 'a) a (sys::%unbound))))
  ...)
--8<---------------cut here---------------end--------------->8---


wdyt? thanks.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://mideasttruth.com
http://honestreporting.com http://memri.org http://iris.org.il
Rottweiler puppy for sale, eats everything, loves children.

Re: some ideas for multithreading support

[<don...@is...>]

 > Fully agree - no crashes.
 > However I think it's ok to have undefined behaviour (not a crash) when
 > multiple threads modify the same resource without synchronisation.

I'm not sure what you mean above by undefined behavior.
I'd hope that after two threads do something like
 (setf a 3)  and  (setf a 7)
we could claim that a is either 3 or 7.
And that all other storage locations are unchanged.
That's what I'd call nondeterministic rather than undefined.
But the same would be true if both setf's were protected by the
same lock.

So again, I think the task is to either define what the possible
outcomes are of unlocked simultaneous access if those outcomes
are all considered reasonable, or if not, introduce locking and
describe what outcomes are possible with the locking and also any 
possible consequences of the locking that might affect the user.
For instance, I hope hash table iteration could be described in 
terms of all of the keys in the table at the beginning plus all 
those added during the iteration being ordered somehow, and the 
iteration visiting them in that order (along with the associated
values at the time the key is visited), but skipping those that 
were not in the table (either cause they were deleted earlier 
or not yet added) at the time the iteration reaches their position.  

(BTW, I think that the statement: 
 the iteration visits once every key that was in the table at the 
 beginning of the iteration but never deleted during the iteration, 
 never visits any key that was not in the table at the beginning 
 and not added during the iteration, and visits either once or zero
 times those that were added or deleted during the iteration
is implied by that but fails to guarantee some desired ordering 
properties guaranteed by the more operational description.)

Re: some ideas for multithreading support

[Vladimir T. <vtz...@gm...>]

On Fri, Dec 8, 2017 at 1:17 AM, Bruno Haible <br...@cl...> wrote:

> Hi all,
>
> In 2008-2012, I couldn't give much input or feedback regarding the
> multithreading implementation. But now, here are 3 ideas for improvement
> that I collected over the last few months.
>
> Vladimir: I appreciate a lot your hard work in this area. Especially the
> GC and signal handling changes are among the most difficult things any
> hacker
> can tackle.
>
>   ----------------------------------------------------------------------
> 1) The philosophy of clisp and the philosophy of multithread support.
>
> I find that it is a must that:
>
>    Simultaneous access to the same object from different threads must,
>    by default, lead to an error instead of a crash.
>

I don't think this can be enforced without severely hurting the
performance. Also as Don notes "What qualifies as an object?"
There are legit cases for resource sharing.


> Why?
>
>   * This is the basic philosophy of clisp: Simple programming mistakes
>     lead to errors, not crashes. (And with decent error messages, in most
>     cases.)
>

Fully agree - no crashes.
However I think it's ok to have undefined behaviour (not a crash) when
multiple threads modify the same resource without synchronisation.



>     While some implementations that compile to native code have default
>     optimization settings that will allow a 'dotimes' loop, for example,
>     to crash if the number is a bignum, clisp's philosophy implies that
>     it does not optimize away type checks if that could lead to a crash.
>
>     The point of keeping the type information on *all* objects, at runtime,
>     is to be able to diagnose a type error in *all* situations. This is
>     a distinguishing feature of clisp.
>
>   * The philosophy of C, on the opposite side, is: Simple programming
>     mistakes lead to crashes. This is the consequence of maximum
> optimization.
>     When you program in C, you know that every time you make a small
> mistake,
>     you risk a crash.
>
>   * [1]: "If you want to shoot yourself, it is your responsibility to wear
> armor."
>     This means that the philosophy of the multithreading support currently
>     is the C philosophy.
>
>   * I wish to have multithreading support with the clisp philosophy, not
> the
>     C philosophy.
>
> How to implement it?
>
> *Every object access* will have to be protected by taking a lock.
>
> For example,
>
>   LISPFUNNR(car,1)
>   {
>     VALUES1(car(popSTACK()));
>   }
>
> will be rewritten to
>
>   LISPFUNNR(car,1)
>   {
>     var object argument = popSTACK();
>     RDLOCK(object);
>     var object result = car(argument);
>     RDUNLOCK(object);
>     VALUES1(result);
>   }
>
> RDLOCK(object) may signal an error:
>   "The object ~S is currently locked by thread ~S."
>
> To this effect, *every object* will have, in its header, information
> about which thread is currently holding a write-lock on the object, or
> how many objects are holding a read-lock on the object. I believe this
> can be done through a 32-bits field. (This field contains less information
> than a "mutex" or "lock" in usual OS programming. Here RDLOCK will never
> wait: it will either signal an error or proceed. It will never put the
> current process in a "wait queue".)
>
> There is literature that explains how to implement efficient locking,
> e.g. "biased locking" [2].
>
>   ----------------------------------------------------------------------
> 2) The set of target platforms.
>
> At the lowest level, where clisp uses C functions from libc, a multithread
> enabled clisp must only use multithread-safe function from libc. For
> example, readdir_r instead of readdir. Except in cases where we can
> guarantee that it's OK.
>
> I don't know what the status on this task is, but I expect that this will
> severely limit the set of target platforms for multithreading to glibc and
> few other systems.
>
>   ----------------------------------------------------------------------
> 3) Making use of standardized facilities
>
> Since Vladimir wrote the multithreading support, standards have caught
> up:
>   * The '__thread' storage class is now also supported on Windows [3].
>   * <stdatomic.h> (or std::atomic in C++) provides portable support for
>     atomics.
>
> I expect these facilities to be better optimized than what we could roll
> on our own. Therefore it will be interesting to see to which extent we can
> make use of them.
>

__thread storage class has wider support on 'modern' platforms and we can
clean up the codebase from USE_CUSTOM_TLS if we decide to abandon support
for older ones. I don't think there are other important changes we can
benefit from.


> This too will limit the set of target platforms. In the end, I expect
> that only modern glibc, Windows, and macOS will be left as target platforms
> for multithreading.
>
>   ----------------------------------------------------------------------
>
> All this is food for the future. No hurry.
>
> Please give yourself 24 hours of thinking before replying. [4]
>
> Bruno
>
> [1] https://clisp.sourceforge.io/impnotes/mt.html#mt-mutable
> [2] https://www.cs.princeton.edu/picasso/mats/HotspotOverview.pdf
> [2] https://msdn.microsoft.com/en-us/library/6yh4a9k1.aspx
> [4] http://phk.freebsd.dk/sagas/bikeshed.html
>
>

Re: pipefail

[Blake M. <bl...@mc...>]

On Fri, Dec 15, 2017 at 8:54 AM, Bruno Haible <br...@cl...> wrote:

> .....
>
> This is what I did and pushed.
>

That fixed the problem for me.  Thanks!

Blake

Re: Compiling on Android

[Jean L. <bu...@gn...>]

On Fri, Dec 15, 2017 at 10:37:23AM +0100, Bruno Haible wrote:
> Hi Jean Louis,
> 
> > Replicant is liberated Android
> > system. http://replicant.us and that means that
> > with the development of Replicant, we will have
> > free phones very soon, as right now not every
> > mobile device is supported by Replicant.
> 
> Unfortunately, I don't see Replicant, nor Free Phones, becoming mainstream
> any time soon. What kinds of phones with free
> operating systems do exist? [1]

Alright, it means it is not worth effort to try to
compile it on Android. But maybe there is not much
work to do?

Many software packages are compiling without
problems. Did you try using Termux? PicoLisp and
tinyscheme, they are there. GNU Emacs is there.

See the list of packages that are installable
through Termux.

Windows is also not "free" but CLISP runs on
it. Before two days I have introduced somebody to
it.

-- 
Jean Louis

Re: pipefail

[Bruno H. <br...@cl...>]

Hi Sam,

> We use pipes in configure:
> ${MAKE} 2>&1 | tee cbcstep2.log
> and this breaks "set -e" without "set -o pipefail" which is not
> supported by dash (/bin/sh on linux).

Yes, /bin/sh happens to be 'dash' on Ubuntu Linux.

> How should we handle this?
> 
> 1. "All platforms we care about have bash or ksh or zsh which support
>    pipefail" (is it really true?)
>    Detect which one is available and use it.
>    The problem is that this is in the top-level configure.

It is too much annoyance to ask the user to run "bash configure", or
to use respawn logic similar to the one in autoconf-generated configure
files. So, better avoid the unportable 'pipefail' feature.

> 2. Use http://cfajohnson.com/shell/cus-faq-2.html#Q11 .

Interesting to see that there is a portable solution to this. But it
unreadable as hell, this game with file descriptors 3 and 4.

>    I would much prefer if it were in gnulib and we merely sourced it --
>    could you please add it there?

gnulib is, so far, not a library of shell functions (except for the
tests/init.sh meant for use in test suites).

And fortunately we don't need the general solution here. Since each of
the interesting commands is sending its output to a log file, we can send
the exit status to the log file as well.

This is what I did and pushed.

Also, when doing "make -f Makefile.devel multibuild-...", it is useful
when the cbcstep3.log and cbcstep4.log files are present even when cbcstep2.log
failed. This allows me to get a quick overview of all build variants through
"ls -l build-*/cbcstep3.log". I restored this behaviour as well.

Bruno

Re: Compiling on Android

[Bruno H. <br...@cl...>]

Hi Jean Louis,

> Replicant is liberated Android
> system. http://replicant.us and that means that
> with the development of Replicant, we will have
> free phones very soon, as right now not every
> mobile device is supported by Replicant.

Unfortunately, I don't see Replicant, nor Free Phones, becoming mainstream
any time soon. What kinds of phones with free operating systems do exist? [1]
* Hardware built by large companies, OS provided by large companies:
  - Only Samsung with Tizen, but it is no longer being sold.
* Hardware built by large companies, OS provided by hackers:
  - Replicant. But all models after the original Galaxy S3 (from 2012)
    have a highly integrated modem chip from Qualcomm that contains a universal
    backdoor and therefore cannot be enabled in Replicant [2].
* Hardware built by small companies: does not have the integration level
  and therefore not the expected combination of weight + battery time
  that other phones have. Additionally, the UI part of the OS will be
  the major challenge.

[1] https://en.wikipedia.org/wiki/List_of_open-source_mobile_phones
[2] https://redmine.replicant.us/projects/replicant/wiki/ReplicantStatus

> I am not familiar with Android stuff, and I did
> not refer to APK if you mean to those packages for
> Android GUI.
> 
> Myself, I am using https://termux.com/ Termux on
> Android, and there are multiple building packages
> and applications already compiled.
> 
> I am using Termux over SSH and through it, I have
> access to GNU packages, and other software.

OK, but that doesn't mean building clisp "for Android". It means
building clisp for use within a specific text-mode emulation app
on Android, in a way that cannot ever be useful for creating UI apps
on Android. I won't invest time in it.

You said compiling "on Android"; technically this is correct, but
nevertheless misleading.

> Then I got stuck in various hard coded /bin/sh within
> CLISP configuration.

This platform should be similar enough to Unix systems that any
single person can do this port in a short time, with some #ifs in
the source code. Good luck!

IMO a much better starting point for doing Lisp on Android is Clojure,
because it's based on Java and therefore integrates with the UI libraries
of Android. [3][4][5]

[3] http://clojure-android.info/
[4] https://github.com/alexander-yakushev/events/blob/master/tutorial.md
[5] http://www.lambdacat.com/clojure-on-android-overview/

Bruno

Re: Compiling on Android

[Bruno H. <br...@cl...>]

Pascal Bourguignon wrote:
> So an Android application could fork a clisp process, and communicate with it.
> The kernel is Linux, so there is definitely stdin and stdout and everything.

Like, for example, TeXmacs can use Maxima (running in gcl or clisp) [1]?

This would be a possible use-case of clisp indeed. However, it's a very particular
software architecture that few packages are likely to adopt.

Bruno

[1] http://www.texmacs.org/tmweb/plugins/cas.en.html#maxima

Re: pipefail

[Sam S. <sd...@gn...>]

Bruno,

We use pipes in configure:
${MAKE} 2>&1 | tee cbcstep2.log
and this breaks "set -e" without "set -o pipefail" which is not
supported by dash (/bin/sh on linux).

How should we handle this?

1. "All platforms we care about have bash or ksh or zsh which support
   pipefail" (is it really true?)
   Detect which one is available and use it.
   The problem is that this is in the top-level configure.

2. Use http://cfajohnson.com/shell/cus-faq-2.html#Q11 .
   I would much prefer if it were in gnulib and we merely sourced it --
   could you please add it there?

Thanks

> * Blake McBride <oy...@zp...zr> [2017-12-12 21:55:54 -0600]:
>
> Okay.  Thanks.  I got the following from a clean checkout:
>
> $ sh -vx configure with-gcc
> ...
> + set -o pipefail
> $



-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://camera.org
http://iris.org.il http://www.dhimmitude.org http://islamexposedonline.com
Just because you're paranoid doesn't mean they AREN'T after you.

Re: Compiling on Android

[Jean L. <bu...@gn...>]

Dear Bruno,

I am glad that you point out to freedom. I am
aware of it. Yet, GNU made evolution from Unix to
GNU/Hurd and later to GNU/Linux and so other
operating systems became free, such as Minix.

In that sense, having CLISP prepared in its
configure script in such manner that it may build
on Android would further contribute to liberation
of Android system and recognition of free
software.

Replicant is liberated Anroid
system. http://replicant.us and that means that
with the development of Replicant, we will have
free phones very soon, as right now not every
mobile device is supported by Replicant.

On Wed, Dec 13, 2017 at 02:37:28AM +0100, Bruno Haible wrote:
> It's not only about compiling it. You would also
> need to rethink the I/O system (no stdin,
> stdout), provide bindings to the (huge) set of
> Android APIs, and integrate with the toolchain
> that produces APKs.

I am not familiar with Android stuff, and I did
not refer to APK if you mean to those packages for
Android GUI.

Myself, I am using https://termux.com/ Termux on
Android, and there are multiple building packages
and applications already compiled.

I am using Termux over SSH and through it, I have
access to GNU packages, and other software.

There is tinyscheme and Pico lisp, and GNU Emacs
is there too. Somehow they compile it.

I could build libsigsegv without problems. Then I
got stuck in various hard coded /bin/sh within
CLISP configuration.

> And what would be the point? How would this help
> making Android more free [1] (less surveillant,
> less controlled by Google) within a reasonable
> time frame?

Just as you see above: Replicant is being
developed to liberate the devices. If you
remember, there was no free BIOS, today we have it
at least on several types of computers, thanks to
Libreboot.

So when CLISP can compile on Android/Replicant, it
will be used on Replicant and Android, until full
solutions are found for Android to be
liberated.

I just don't think it is so difficult. I need
development version as tar.gz to try it again,
last time I tried with stable version.


-- 
Jean Louis

Re: Build problem on 64 bit Linux

[Blake M. <bl...@mc...>]

Okay.  Thanks.  I got the following from a clean checkout:

$ sh -vx configure with-gcc
...
esac
+ STACK_LIMIT=8192
stacksizelimit=`ulimit -s 2>/dev/null || :` # cygwin /bin/sh ulimit is
broken
+ ulimit -s
+ stacksizelimit=8192
# need 3 separate test calls because of "integer expression expected" errors
# when $stacksizelimit is "" or "unlimited" (no short-circuiting!)
set +e;
+ set +e
test -z "$stacksizelimit" || { test "$stacksizelimit" != unlimited && test
"$stacksizelimit" -lt ${STACK_LIMIT}; }
+ test -z 8192
+ test 8192 != unlimited
+ test 8192 -lt 8192
STACK_TOO_SMALL=$?      # 0=true => need to reset; 1=false => big enough
+ STACK_TOO_SMALL=1
# Ensure exit on failure in pipes (... | tee ...) below.
# This works with bash, ksh and zsh.
set -o pipefail 2>/dev/null
+ set -o pipefail
$



On Mon, Dec 11, 2017 at 9:49 PM, Sam Steingold <sd...@gn...> wrote:

> > * Blake McBride <oy...@zp...zr> [2017-12-11 20:37:32 -0600]:
> >
> > Every time I run it with bash -vx the whole thing works and I can build.
> > Every time I run ./configure without bash -xv it fails as I described.
>
> my fault.
> use "sh" instead of "bash".
> looks like we use an extension of bash which is not supported by stock
> sh.
> (I know you are on linux and sh is a link to bash, but when you run it
> as sh instead of bash is behaves differently)
>
> Thanks.
>
> --
> Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
> http://steingoldpsychology.com http://www.childpsy.net http://camera.org
> http://no2bds.org http://thereligionofpeace.com http://www.dhimmitude.org
> My name is Deja Vu. Have we met before?
>

Re: Compiling on Android

[Pascal B. <pj...@in...>]

> On 13 Dec 2017, at 02:37, Bruno Haible <br...@cl...> wrote:
> 
> Hi Jean Louis,
> 
>> I would like to be able to compile CLISP on Android. It would require some
>> changes like /bin/sh to be definable and not hard coded.
>> 
>> Anybody compiled it on Android?
> 
> No. not as far as I know.
> 
> It's not only about compiling it. You would also need to rethink the I/O system
> (no stdin, stdout), provide bindings to the (huge) set of Android APIs, and
> integrate with the toolchain that produces APKs.
> 
> And what would be the point? How would this help making Android more free [1]
> (less surveillant, less controlled by Google) within a reasonable time frame?

Android applications can also run external processes (contrarily to iOS). 
So an Android application could fork a clisp process, and communicate with it.
The kernel is Linux, so there is definitely stdin and stdout and everything.


Then of course, there’s the eventual possibility of compiling clisp as a library to be linked with the Android applications and used thru JNI.
You wouldn’t implement the UI in lisp. 

-- 
__Pascal J. Bourguignon__

Re: Compiling on Android

[Bruno H. <br...@cl...>]

Hi Jean Louis,

> I would like to be able to compile CLISP on Android. It would require some
> changes like /bin/sh to be definable and not hard coded.
> 
> Anybody compiled it on Android?

No. not as far as I know.

It's not only about compiling it. You would also need to rethink the I/O system
(no stdin, stdout), provide bindings to the (huge) set of Android APIs, and
integrate with the toolchain that produces APKs.

And what would be the point? How would this help making Android more free [1]
(less surveillant, less controlled by Google) within a reasonable time frame?

Bruno

[1] https://www.gnu.org/philosophy/android-and-users-freedom.en.html

Compiling on Android

[Jean L. <bu...@gn...>]

I would like to be able to compile CLISP on Android. It would require some changes like /bin/sh to be definable and not hard coded.

Anybody compiled it on Android?

Jean

Re: Build problem on 64 bit Linux

[Sam S. <sd...@gn...>]

> * Blake McBride <oy...@zp...zr> [2017-12-11 20:37:32 -0600]:
>
> Every time I run it with bash -vx the whole thing works and I can build.
> Every time I run ./configure without bash -xv it fails as I described.

my fault.
use "sh" instead of "bash".
looks like we use an extension of bash which is not supported by stock
sh.
(I know you are on linux and sh is a link to bash, but when you run it
as sh instead of bash is behaves differently)

Thanks.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://camera.org
http://no2bds.org http://thereligionofpeace.com http://www.dhimmitude.org
My name is Deja Vu. Have we met before?

Re: Build problem on 64 bit Linux

[Blake M. <bl...@mc...>]

Thanks for the help!!

Every time I run it with bash -vx the whole thing works and I can build.
Every time I run ./configure without bash -xv it fails as I described.

Even with just "bash ./configure with-gcc" it works.  The bash command
creates a new environment.  There must be something in my regular
environment that is causing the problem.

How else can I figure out the problem?

Thanks.

Blake


On Mon, Dec 11, 2017 at 8:59 AM, Sam Steingold <sd...@gn...> wrote:

> > * Blake McBride <oy...@zp...zr> [2017-12-10 18:04:51 -0600]:
> >
> > Still a problem.  Here you go:
> >
> > $ rm -rf *
> > $ ls -l
> > total 0
> > $ hg update -C
> > 1383 files updated, 0 files merged, 0 files removed, 0 files unresolved
> > $ ls
> > ANNOUNCE    DEPENDENCIES  HACKING          Makefile.in  SUMMARY
> version.sh
> > benchmarks  doc           INSTALL          modules      tests
> win32msvc
> > configure   emacs         INSTALL.windows  sacla-tests  unix
> > COPYRIGHT   GNU-GPL       Makefile.devel   src          utils
> > $ ./configure build-with-gcc
> > ...
> > config.status: executing depfiles commands
> > config.status: executing libtool commands
> > config.status: executing po-directories commands
> > config.status: creating po/POTFILES
> > config.status: creating po/Makefile
> > Configure findings:
> >   FFI:        yes (user requested: default)
> >   readline:   yes (user requested: default)
> >   libsigsegv: yes
> > $ cd build-with-gcc
> > $ make
> > make: *** No targets specified and no makefile found.  Stop.
> > $ ls
> > config.cache  config.log     floatparam.h  intparam.h  makemake  stamp-h1
> > config.h      config.status  gllib         libtool     po
>
> When I repeat what you did, my output ended with
>
> --8<---------------cut here---------------start------------->8---
> ...
> Configure findings:
>   FFI:        no, consider installing GNU libffcall (user requested:
> default)
>   readline:   yes (user requested: default)
>   libsigsegv: yes
> ./makemake  > Makefile
> Configured with: --prefix=/Library/Developer/CommandLineTools/usr
> --with-gxx-include-dir=/usr/include/c++/4.2.1
> cp -p ../src/cfgunix.lisp config.lisp
> chmod +w config.lisp
> echo '(setq *clhs-root-default* "http://www.ai.mit.edu/
> projects/iiip/doc/CommonLISP/HyperSpec/")' >> config.lisp
>
> To continue building CLISP, the following commands are recommended
>   (cf. unix/INSTALL step 4 ff):
>     cd build-with-gcc
>     /Users/sds/src/emacs/trunk/build/lib-src/emacsclient config.lisp
>     make
> # CLISP self-test:
>     make check
> # Test non-portable features and modules
>     make extracheck mod-check
> --8<---------------cut here---------------end--------------->8---
>
> IOW, your "./configure" probably failed (try "./configure || echo failed").
> The question is why.
> Could you please run it as "bash -vx configure"?
> You will see a lot of output, and the end will, hopefully, shed some
> light on what the problem is.
>
> Thanks.
>
> --
> Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
> http://steingoldpsychology.com http://www.childpsy.net
> http://think-israel.org
> http://www.memritv.org http://islamexposedonline.com
> http://www.dhimmitude.org
> The more project management you do, the less likely your project is to
> succeed.
>

Re: Build problem on 64 bit Linux

[Sam S. <sd...@gn...>]

> * Blake McBride <oy...@zp...zr> [2017-12-10 18:04:51 -0600]:
>
> Still a problem.  Here you go:
>
> $ rm -rf *
> $ ls -l
> total 0
> $ hg update -C
> 1383 files updated, 0 files merged, 0 files removed, 0 files unresolved
> $ ls
> ANNOUNCE    DEPENDENCIES  HACKING          Makefile.in  SUMMARY  version.sh
> benchmarks  doc           INSTALL          modules      tests    win32msvc
> configure   emacs         INSTALL.windows  sacla-tests  unix
> COPYRIGHT   GNU-GPL       Makefile.devel   src          utils
> $ ./configure build-with-gcc
> ...
> config.status: executing depfiles commands
> config.status: executing libtool commands
> config.status: executing po-directories commands
> config.status: creating po/POTFILES
> config.status: creating po/Makefile
> Configure findings:
>   FFI:        yes (user requested: default)
>   readline:   yes (user requested: default)
>   libsigsegv: yes
> $ cd build-with-gcc
> $ make
> make: *** No targets specified and no makefile found.  Stop.
> $ ls
> config.cache  config.log     floatparam.h  intparam.h  makemake  stamp-h1
> config.h      config.status  gllib         libtool     po

When I repeat what you did, my output ended with

--8<---------------cut here---------------start------------->8---
...
Configure findings:
  FFI:        no, consider installing GNU libffcall (user requested: default)
  readline:   yes (user requested: default)
  libsigsegv: yes
./makemake  > Makefile
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
cp -p ../src/cfgunix.lisp config.lisp
chmod +w config.lisp
echo '(setq *clhs-root-default* "http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/")' >> config.lisp

To continue building CLISP, the following commands are recommended
  (cf. unix/INSTALL step 4 ff):
    cd build-with-gcc
    /Users/sds/src/emacs/trunk/build/lib-src/emacsclient config.lisp
    make
# CLISP self-test:
    make check
# Test non-portable features and modules
    make extracheck mod-check
--8<---------------cut here---------------end--------------->8---

IOW, your "./configure" probably failed (try "./configure || echo failed").
The question is why.
Could you please run it as "bash -vx configure"?
You will see a lot of output, and the end will, hopefully, shed some
light on what the problem is.

Thanks.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://think-israel.org
http://www.memritv.org http://islamexposedonline.com http://www.dhimmitude.org
The more project management you do, the less likely your project is to succeed.

Re: nightly build error fixed

[Jean L. <bu...@gn...>]

On Sun, Dec 10, 2017 at 11:50:10PM +0000, Don Cohen wrote:
> Sam Steingold writes:
>  > > * Don Cohen <qba...@vf...3-vap.pbz> [2017-12-10 05:22:15 +0000]:
>  > >
>  > >  > ./lisp.run -B . -N locale -E UTF-8 -Emisc 1:1 -Epathname 1:1 -norc -q
>  > >  > -M lispinit.mem -x '(truename (make-stream :output))'
>  > >  > *** - TRUENAME: File #P"/dev/pts/3" does not exist
>  > >  > 
>  > >  > make: *** [Makefile:2237: check-script] Error 1
>  > >  > admin [ /sources/gnu/clisp-clisp/build-2017-12-09-15:28:18-Saturday
>  > >  > ]$ ls /dev/pts/ptmx
>  > 
>  > what is your platform and setup?
>  > (linux? cron as root? configure options?)

GNU system with Linux kernel

My configure options are following:

ulimit -s 16384 && ./configure --prefix=/package/prog/clisp-`date +'''%F-%T-%A'''` --with-threads=POSIX_THREADS --enable-portability --with-module=asdf --with-module=berkeley-db --with-module=bindings/glibc --with-module=clx/new-clx --with-module=dbus --with-module=editor --with-module=fastcgi --with-module=gdbm --with-module=gtk2 --with-module=pcre --with-module=postgresql --with-module=rawsock --with-module=zlib build-`date +'''%F-%T-%A'''`

I do not use cron as root, just as user.

Maybe it is up to me, I am using the small
insertable camera stick for development of the
whole system, then I am pulling it with rsync onto
the hard disk. So it could be that I did something
wrong, maybe I did not create all /dev/pts/1-4
devices on that disk, even it should be there it
is devpts file system.

Jean

Re: some ideas for multithreading support

[<don...@is...>]

I'm not as familiar with the details as the others commenting on this
thread, so maybe you could give me some ideas where to look for
answers to a few questions.

 > > *Every object access* will have to be protected by taking a lock.

What qualifies as an object?
A fixnum?  That's immutable, right?  So I expect no lock is needed.
How about a cons cell?
Under what circumstances could two threads accessing a cons cell 
cause a crash or any illegal lisp data?
I can see that garbage collection is a special case, but I thought
it already was handled in some way (which I'd be happy to learn more
about).
If two threads are not in GC and they are only either reading the
car or cdr of a cons cell or replacing it with some legitimate value,
and these are atomic operations, I don't see the problem.
What am I missing?
(Is a pointer on your stack something you can write into a cons cell?
If so, this seems as much a problem without MT as with it.)

I imagine that other objects are similar to cons cells in the sense
above, except that they might have some non-atomic operations that
have to be protected by locks.
Is that all correct?  If so, the issue becomes identifying the 
operations that have to be protected.

 > > I don't know what the status on this task is, but I expect that this
 > > will severely limit the set of target platforms for multithreading to
 > > glibc and few other systems.

I'm guessing that the glibc issue is that different libraries used
by clisp offer the "same" facilities but they're thread safe in some
libraries but not others.  And glibc tends to be more thread safe than
the others.  Is that correct?  So you might have to stop using some
libraries or replace certain operations in them with thread safe versions.
Sounds pretty painful, I admit.  Not to mention maintenance as these
libraries evolve.

 > > This too will limit the set of target platforms. In the end, I expect
 > > that only modern glibc, Windows, and macOS will be left as target
 > > platforms for multithreading.
 > 
 > which is sad because one of the salient features of clisp is
 > portability.

However it's not so bad when some feature that's not so portable was
not part of the standard to begin with.  Clisp has other platform
dependent features.

Re: Build problem on 64 bit Linux

[Blake M. <bl...@mc...>]

Still a problem.  Here you go:

$ rm -rf *
$ ls -l
total 0
$ hg update -C
1383 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ ls
ANNOUNCE    DEPENDENCIES  HACKING          Makefile.in  SUMMARY  version.sh
benchmarks  doc           INSTALL          modules      tests    win32msvc
configure   emacs         INSTALL.windows  sacla-tests  unix
COPYRIGHT   GNU-GPL       Makefile.devel   src          utils
$ ./configure build-with-gcc
...
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing po-directories commands
config.status: creating po/POTFILES
config.status: creating po/Makefile
Configure findings:
  FFI:        yes (user requested: default)
  readline:   yes (user requested: default)
  libsigsegv: yes
$ cd build-with-gcc
$ make
make: *** No targets specified and no makefile found.  Stop.
$ ls
config.cache  config.log     floatparam.h  intparam.h  makemake  stamp-h1
config.h      config.status  gllib         libtool     po




On Sun, Dec 10, 2017 at 3:33 PM, Sam Steingold <sd...@gn...> wrote:

> Hi,
>
> > * Blake McBride <oy...@zp...zr> [2017-12-08 23:38:59 -0600]:
> >
> > I'm trying to build CLISP and running into the following problem:
> >
> > $ hg pull
> > pulling from http://hg.code.sf.net/p/clisp/clisp
> > searching for changes
> > no changes found
> > $ ./configure
>
> You might want to start with clisp/unix/INSTALL.
> We recommend building in a separate directory.
> (to be honest, I don't even remember when I last built clisp in src)
>
> --
> Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
> http://steingoldpsychology.com http://www.childpsy.net http://camera.org
> http://honestreporting.com http://www.dhimmitude.org http://jij.org
> If you cannot improve on silence, keep it.
>

Re: nightly build error fixed

[<don...@is...>]

Sam Steingold writes:
 > > * Don Cohen <qba...@vf...3-vap.pbz> [2017-12-10 05:22:15 +0000]:
 > >
 > >  > ./lisp.run -B . -N locale -E UTF-8 -Emisc 1:1 -Epathname 1:1 -norc -q
 > >  > -M lispinit.mem -x '(truename (make-stream :output))'
 > >  > *** - TRUENAME: File #P"/dev/pts/3" does not exist
 > >  > 
 > >  > make: *** [Makefile:2237: check-script] Error 1
 > >  > admin [ /sources/gnu/clisp-clisp/build-2017-12-09-15:28:18-Saturday
 > >  > ]$ ls /dev/pts/ptmx
 > 
 > what is your platform and setup?
 > (linux? cron as root? configure options?)

note that the above problem was reported by Jean Louis and did
not happen to me -- so I think you're asking for HIS platform, etc.

the stuff below, however, is mine

 > >  Form: (HANDLER-CASE (LETF ((*CURRENT-LANGUAGE* 'FRENCH)) (LIST (STRING=
 > > "À bientôt!" (SYSTEM::TEXT "Bye.")) (STRING= *CURRENT-LANGUAGE*
 > > "FRANÇAIS"))) (ERROR (E) (PRINC-ERROR E) '(T T)))
 > >  CORRECT: (T T)
 > >  CLISP  : (NIL T)
 > >  Differ at position 0: T vs NIL
 > >  CORRECT: (T T)
 > >  CLISP  : (NIL T)
 > 
 > are you building --without-unicode ?

I don't think so, and in fact :UNICODE does appear in *features*,
and for that matter FRANCAIS (with the extra mark on the C) does
appear in the *CURRENT-LANGUAGE* test.

 > >  Form: (WITH-OPEN-FILE (COPY S) (STREAMP COPY))
 > >  CORRECT: T
 > >  CLISP  : ERROR
 > >  OS-FILE-ERROR(13): Permission denied
 > >  OUT:
 > >  "[OS-FILE-ERROR]: OS-FILE-ERROR(13): Permission denied
 > >  "
 > >
 > >
 > >  Form: (LET ((*REOPEN-OPEN-FILE* NIL)) (WITH-OPEN-FILE (COPY S
 > > :DIRECTION :OUTPUT) (STREAMP COPY)))
 > >  CORRECT: T
 > >  CLISP  : ERROR
 > >  OS-FILE-ERROR(13): Permission denied
 > >  OUT:
 > >  "[OS-FILE-ERROR]: OS-FILE-ERROR(13): Permission denied
 > >  "
 > 
 > I think this is the same failure as the TRUENAME above.

Even though the truename above did not fail for me.

So maybe you do want my platform and setup...
yes linux, fedora, fc24.x86_64
make check run by user don, not even from cron
Configure and make were done from cron but also as user don.
./configure CC='gcc -m64' --disable-maintainer-mode --with-debug --with-module=rawsock build-dir 

Note that if I run clisp as root then:
[1]> (WITH-OPEN-FILE (COPY (make-stream :INPUT)) (STREAMP COPY))
T

There are some things that are expected to work as root but not as 
other users (rawsock I think has some).  It was not clear to me how you
wanted to test such things.  I don't know why this should or should not
be such a thing.  There are probably legitimate choices made by different
OS versions on things like permission of users to access /dev/pts/9

Break 1 [12]> (truename (make-stream :OUTPUT))
#P"/dev/pts/9"

$ ls -l /dev/pts/9
crw--w----. 1 root tty 136, 9 Dec 10 15:48 /dev/pts/9

Re: some ideas for multithreading support

[Sam S. <sd...@gn...>]

Hi,

> * Bruno Haible <oe...@py...t> [2017-12-08 00:17:12 +0100]:
>
>    ...an error instead of a crash.

always.

> *Every object access* will have to be protected by taking a lock.

This may turn out to be far too expensive.
We might want to consider thread-local heap, similar to generational GC:
the most recent generation which is collected often may be thread-local
and require an explicit operation to share to a different thread and
require no lock for read/write.
on the other hand, we might want to share heap explicitly only: i.e.,
all heap is thread-local and there is an explicit (recursive/deep) copy
to shared (read-only?) heap.
I.e., to modify shared data, a thread copies it locally, modifies, and
copies back to shared heap.


> For example,
>
>   LISPFUNNR(car,1)
>   {
>     VALUES1(car(popSTACK()));
>   }
>
> will be rewritten to
>
>   LISPFUNNR(car,1)
>   {
>     var object argument = popSTACK();
>     RDLOCK(object);
>     var object result = car(argument);
>     RDUNLOCK(object);
>     VALUES1(result);
>   }

I think simultaneous reading is no problem.
I think writers should get an exclusive lock and readers a shared lock.

> RDLOCK(object) may signal an error:
>   "The object ~S is currently locked by thread ~S."

1. The error must have a "retry" restart.
2. The kind of lock (read/shared vs write/exclusive) should also be
specified.

> ...readdir_r instead of readdir.

note that readdir_r is deprecated.
https://www.gnu.org/software/libc/manual/html_node/Reading_002fClosing-Directory.html

> I don't know what the status on this task is, but I expect that this
> will severely limit the set of target platforms for multithreading to
> glibc and few other systems.

... which will severely limit the usability of multithreaded clisp.

> 3) Making use of standardized facilities
>
> I expect these facilities to be better optimized than what we could
> roll on our own. Therefore it will be interesting to see to which
> extent we can make use of them.

indeed!

> This too will limit the set of target platforms. In the end, I expect
> that only modern glibc, Windows, and macOS will be left as target
> platforms for multithreading.

which is sad because one of the salient features of clisp is
portability.

Thanks!

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net
http://jij.org http://no2bds.org http://americancensorship.org
Which should be served first: the chicken salad or the egg salad?

Re: nightly build error fixed

[Sam S. <sd...@gn...>]

> * Don Cohen <qba...@vf...3-vap.pbz> [2017-12-10 05:22:15 +0000]:
>
>  > ./lisp.run -B . -N locale -E UTF-8 -Emisc 1:1 -Epathname 1:1 -norc -q
>  > -M lispinit.mem -x '(truename (make-stream :output))'
>  > *** - TRUENAME: File #P"/dev/pts/3" does not exist
>  > 
>  > make: *** [Makefile:2237: check-script] Error 1
>  > admin [ /sources/gnu/clisp-clisp/build-2017-12-09-15:28:18-Saturday
>  > ]$ ls /dev/pts/ptmx

what is your platform and setup?
(linux? cron as root? configure options?)

>  Form: (HANDLER-CASE (LETF ((*CURRENT-LANGUAGE* 'FRENCH)) (LIST (STRING=
> "À bientôt!" (SYSTEM::TEXT "Bye.")) (STRING= *CURRENT-LANGUAGE*
> "FRANÇAIS"))) (ERROR (E) (PRINC-ERROR E) '(T T)))
>  CORRECT: (T T)
>  CLISP  : (NIL T)
>  Differ at position 0: T vs NIL
>  CORRECT: (T T)
>  CLISP  : (NIL T)

are you building --without-unicode ?


>  Form: (WITH-OPEN-FILE (COPY S) (STREAMP COPY))
>  CORRECT: T
>  CLISP  : ERROR
>  OS-FILE-ERROR(13): Permission denied
>  OUT:
>  "[OS-FILE-ERROR]: OS-FILE-ERROR(13): Permission denied
>  "
>
>
>  Form: (LET ((*REOPEN-OPEN-FILE* NIL)) (WITH-OPEN-FILE (COPY S
> :DIRECTION :OUTPUT) (STREAMP COPY)))
>  CORRECT: T
>  CLISP  : ERROR
>  OS-FILE-ERROR(13): Permission denied
>  OUT:
>  "[OS-FILE-ERROR]: OS-FILE-ERROR(13): Permission denied
>  "

I think this is the same failure as the TRUENAME above.


> I was not aware of either LETF or *CURRENT-LANGUAGE* before.

https://clisp.sourceforge.io/impnotes/macros3.html#letf
https://clisp.sourceforge.io/impnotes/i18n.html#language

thanks

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://honestreporting.com
http://camera.org http://jij.org http://www.memritv.org https://jihadwatch.org
The difference between genius and stupidity is that genius has its
limits.

Re: Build problem on 64 bit Linux

[Sam S. <sd...@gn...>]

Hi,

> * Blake McBride <oy...@zp...zr> [2017-12-08 23:38:59 -0600]:
>
> I'm trying to build CLISP and running into the following problem:
>
> $ hg pull
> pulling from http://hg.code.sf.net/p/clisp/clisp
> searching for changes
> no changes found
> $ ./configure

You might want to start with clisp/unix/INSTALL.
We recommend building in a separate directory.
(to be honest, I don't even remember when I last built clisp in src)

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://camera.org
http://honestreporting.com http://www.dhimmitude.org http://jij.org
If you cannot improve on silence, keep it.