clisp-devel — closed list for developers and testers

sourceforge miseries

[Bruno H. <br...@cl...>]

mercurial access to sourceforge is highly unreliable these days again:

$ hg in
comparing with ssh://ha...@hg.../p/clisp/clisp
^Cinterrupted!
$ hg in
remote: abort: there is no Mercurial repository here (.hg not found)!
abort: no suitable response from remote hg!
$ hg in
comparing with ssh://ha...@hg.../p/clisp/clisp
searching for changes
no changes found
$ hg push
pushing to ssh://ha...@hg.../p/clisp/clisp
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: abort: Transport endpoint is not connected
remote: transaction abort!
remote: rollback failed - please run hg recover
abort: unexpected response: empty string

And no report at https://twitter.com/sfnet_ops.

This is not the first time the hg server at sourceforge is acting miserably.

To me, this increases the priority of moving the source code repository
away from sourceforge.

We can consider the following options:

  a) Move it to savannah.gnu.org (yes, savannah supports hg, see
     http://hg.savannah.gnu.org/hgweb/?sort=lastchange ),

  b) Convert it to git and move it to savannah.gnu.org.

  c) Convert it to git and move it to gitlab.com.

Discussion:

- Advantage of a): Less work, can be realized in a short time frame.

- Advantage of b), c): Reduce the entry barrier for young developers
  (they all know how to use git, but hardly anyone uses mercurial).

Bruno

Re: GSoC Free-Lock Hash Table, Multiple library bindings, etc.

[Javier A. G. T. <jav...@gm...>]

Hi Bruno!

> But a Common Lisp binding to libvirt already exists [1], and I would expect
> that it does not take long to make this binding actually run in clisp. (Maybe
> just 1 week?)
>

Yes, but as I can tell, this library is manually made, on the contrary
python libvirt bindings
are automatically created utilizing the API XML descriptions from
libvirt.[1] Part of the project will involve
to mimic the python lib but better (macros and provably run time
actualization as it seems as cool idea but all deepens on the libvirt
C API updates).
1-2 weeks of overhead.

> So, to make this a valuable GSoC project, you would need to add significant
> functionality on top of it. Just "basic functionality of a cloud orchestrator"
> sounds like too little for me.

Ok, a cloud orchestrator composed by a single controller whit N
compute nodes. For communication between nodes I was thinking on
cl-rabbit, where a subset of the C API  is ready. For database, sql
alchemy like library already found con common lisp utilized on
caveman2 web framework project. As you state, multi thread on clisp is
not on production status,
maybe use user-level multi thread library?

Basic functions regarding the orchestrator (utilizing libvirt):
1.- Creation of instances based on flavors (Basic description of
2.- Power operations over instances (Halt, restart, etc.)
3.- Layer 2 Networking including a subset of a firewall (Port-forwarding)
4.- Instance migration

Added functionalities:
1.-Workloads: Description of the OS image, VCPUs, RAM and a bash or
clisp script for pre-configuration.
2.- key-pairs management.
3.- openstack identity service for authentication using its REST API.
4.- OS images management.
5.- Run-time patches. On Open stack this represents a problem to the
point on having a single project to upgrade from a version to
another.[2]


The controller node will perform the  orchestration of the cloud:
1.- Decide where to create a new instance.
2.- Handle the CRUD for networking, key-pairs, OS images and workloads.
3.- Apply patches to other nodes.
4.- Given access only to authenticated nodes and users.

The compute node will handle the operations given by the controller
delivered by the rabbitmq broker and information founded on the
database.

Deliverables:
libvirt
Patches to cl-rabbit and any other library of needed
controller daemon.
compute daemon.

Benefits to CLISP:
1.- More libraries available.
2.- Incentivize cloud development on an already capable language.
3.- A simpler private cloud application where an smaller organization
can deploy their services based on clisp.

Another idea that it comes to my mind is to create a nova service
clone to run on an already deployed open stack cluster to show of
run-time patches.
And a follower to this project, a network node to support layer 3
networking (Networks defined by software).


Background:
1.- sysadmin newbie from both openstack and ciao cluster.
2.- Done basic school work on SBCL. Relevant to this project: a proof
of concept cloud orchestrator using cl-rabbit and cl-vagrant.

Any critique is welcome in order to produce a proposal draft.
Thanks for your time.


PD: Hopefully I can deliver at least the libvirt library on any case.

Instance: Virtual machine already running.
[1] https://libvirt.org/python.html
[2] https://wiki.openstack.org/wiki/Upgrade-with-minimal-downtime

Re: GSoC compiler work, SICL

[Charles Z. <ka...@be...>]

Hi Bruno, Sam,

> Re 3: clisp's LOOP has a number of optimizations already in the macro.
> It is possible that cleavir adds more optimizations to it, but this would
> not be the major benefit.
By 'loop optimizations', I meant more flow-graph analytic
optimizations, eg. using dominance information on the instruction
graph. Perhaps clisp already does these sorts of optimizations as well
at a higher level. This would fall into the general category of flow
sensitive optimizations, which I envision would be an ambitious but
important part of the SICL integration.

> This is good; however, we should still strive to define the functionality
> of the deliverables. For example, pick the right ones among these:
>   - Make the SICL compiler core run in clisp.
>   - Connect it to the clisp LAP assembler as backend, so that it can be
>     used to create compiled clisp functions.
I agree that getting the SICL compiler core running in clisp would be
the main deliverable; after all, that would be the prerequisite for
adding anything else from Cleavir.

>   - Make the COMPILE function work flawlessly in all cases.
>   - Make the DISASSEMBLE function work flawlessly in all cases.
>   - Make the COMPILE-FILE function work flawlessly in all cases.
>     This implies proper handling of LOAD-TIME-VALUE.

I am not too sure what you mean here. I think that SICL would be
integrated at a level high enough where this would not impact the
bytecode emitter/dissassembler, and hence not disrupt DISASSEMBLE, etc
too much. As you say, once Cleavir is connected to the LAP side of
things, it should be emitting the already existing clisp bytecode.
There are SSA-form and Register-Allocation modules as well in SICL
that work at a lower level, but I think this may be stretching the
scope of the project. In other words, it seems like it is possible to
add all the necessary machinery to support dataflow analysis from SICL
within the existing framework of the clisp bytecode and FASL formats.

>> Calling closures always conses (by calling copy-closure).
>> Can this be eliminated?
>> ...
>> > Scheme-like optimizations.
>>
>> What is that?
>
> This is what I meant. Based on data-flow analysis, the compiler may determine
> that a local function is only used within the dynamic extent of the function
> that defines it. Which means that the closure can be stack-allocated - no
> consing.
I believe SICL has already produced an escape analysis module which is
being actively developed and used with clasp. So yes - better dynamic
extent allocation in general and non-consing closures specifically
would be another possible deliverable. I think it would actually be
much harder to implement general escape analyses without a general
flow graph structure, so getting SICL in might be the 'right' way to
do this.

> Yes, I am willing to mentor you with this kind of project.
Great, I'll be discussing with the SICL people and start fleshing out
a proposal to be submitted by next week.

Thanks,
Charles


On Wed, Mar 21, 2018 at 3:50 PM, Bruno Haible <br...@cl...> wrote:
> Sam wrote:
>
>> I think better handling of local functions in general would be a good
>> improvement and a sizable project.
>
> Yes. Note that this can go into different directions:
>   - Compiling local functions in a better way requires data-flow analysis;
>     this would be one outcome of the SICL work.
>   - Making local functions more debuggable, on the other hand, goes in the
>     opposite direction, as it prevents some optimizations. In my opinion,
>     this is much less work than a GSoC project (only ca. 2-4 weeks).
>
>> Calling closures always conses (by calling copy-closure).
>> Can this be eliminated?
>> ...
>> > Scheme-like optimizations.
>>
>> What is that?
>
> This is what I meant. Based on data-flow analysis, the compiler may determine
> that a local function is only used within the dynamic extent of the function
> that defines it. Which means that the closure can be stack-allocated - no
> consing.
>
> There are surely more optimizations that a good Scheme compiler does, that
> would apply in clisp as well.
>
> Bruno
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> clisp-devel mailing list
> cli...@li...
> https://lists.sourceforge.net/lists/listinfo/clisp-devel



-- 
Class of 2021

Re: GSoC compiler work, SICL

[Sam S. <sd...@gn...>]

Bruno,

> * Bruno Haible <oe...@py...t> [2018-03-21 23:50:04 +0100]:
>
>   - Compiling local functions in a better way requires data-flow analysis;
>     this would be one outcome of the SICL work.
>   - Making local functions more debuggable, on the other hand, goes in the
>     opposite direction, as it prevents some optimizations.

That's what the OPTIMIZE declaration is for!
http://clhs.lisp.se/Body/d_optimi.htm

>     In my opinion,
>     this is much less work than a GSoC project (only ca. 2-4 weeks).

Bruno, not everyone is as smart as you are. Alas.
I don't know about Charles, but for me it would certainly be more than
a month.
Moreover, this is not just implementation - one has to update the docs,
test the performance impact, learn clisp tool chain &c &c.

Thanks.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1561
http://steingoldpsychology.com http://www.childpsy.net https://jihadwatch.org
http://honestreporting.com http://mideasttruth.com http://www.memritv.org
Software is like sex: it's better when it's free.

Re: GSoC compiler work, SICL

[Bruno H. <br...@cl...>]

Sam wrote:

> I think better handling of local functions in general would be a good
> improvement and a sizable project.

Yes. Note that this can go into different directions:
  - Compiling local functions in a better way requires data-flow analysis;
    this would be one outcome of the SICL work.
  - Making local functions more debuggable, on the other hand, goes in the
    opposite direction, as it prevents some optimizations. In my opinion,
    this is much less work than a GSoC project (only ca. 2-4 weeks).

> Calling closures always conses (by calling copy-closure).
> Can this be eliminated?
> ...
> > Scheme-like optimizations.
> 
> What is that?

This is what I meant. Based on data-flow analysis, the compiler may determine
that a local function is only used within the dynamic extent of the function
that defines it. Which means that the closure can be stack-allocated - no
consing.

There are surely more optimizations that a good Scheme compiler does, that
would apply in clisp as well.

Bruno

Re: GSoC compiler work, SICL

[Sam S. <sd...@gn...>]

Hi Bruno & Charles,

> * Bruno Haible <oe...@py...t> [2018-03-21 22:35:39 +0100]:
>
> inlining of local functions,

I think better handling of local functions in general would be a good
improvement and a sizable project.
Inlining them is one thing (btw, they do not have to be inlined _always_).
(and doing it well probably entails removing function-macro-let, after
making sure that this does not cost performance)

Many years ago I added disassembling local functions:

--8<---------------cut here---------------start------------->8---
(defun foo (x z)
  (flet ((add (y) (+ x y)))
    (values (apply #'add z nil) #'add)))
(disassemble 'foo)
(disassemble  (local foo add))
--8<---------------cut here---------------end--------------->8---

How about tracing them too?
This would make debugging CLISP loop much easier (loop is implemented as
a huge function with an insane number of local functions, some of which
are redefined inside other local functions).

Calling closures always conses (by calling copy-closure).
Can this be eliminated?

> data-flow optimizations

e.g., eliminate the first `bar`:

--8<---------------cut here---------------start------------->8---
(defun foo ()
  (labels ((bar () t)) (bar))
  (labels ((bar () nil)) (bar)))
(disassemble 'foo)
--8<---------------cut here---------------end--------------->8---

> Scheme-like optimizations.

What is that?
CLISP already eliminates tail calls.

> Sam, are you willing as well?

Yes.

Thanks.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1561
http://steingoldpsychology.com http://www.childpsy.net
http://www.memritv.org http://mideasttruth.com http://no2bds.org http://jij.org
Bug free software merely has random features.

Re: GSoC compiler work, SICL

[Bruno H. <br...@cl...>]

Hi Charles,

> As GSoC applications have opened and the deadline for submissions is
> approaching, I'd like to reach out again to the clisp developers.

Good that you ask again. (Your first mail came in a period where
I was pretty busy, sorry about that.)

> It seems that the overall evaluation of the feasibility of adding
> native file compilation or ahead of time compilation to machine code
> is that it is way too ambitious. On the other hand, various
> self-contained bugs such as improving local inlining would not be
> enough to form a project on its own. ...
> A good compromise seems to be adding pieces of SICL to clisp, with
> Cleavir being the most useful.

I agree with these evaluations.

> * Probably the easiest piece to add are the various ANSI CL modules,
> such as sequence functions, reader, and loop, among others. Of course,
> this isn't extremely useful given that CLISP already conforms to ANSI
> CL, but there are some novel techniques that Robert Strandh has
> introduced that make it worthwhile to look into replacing some of
> clisp's implementations.

These parts of clisp (sequence functions, reader, and loop, and others)
are pretty well implemented and optimized, in my opinion. You would have
to be *very convincing* about the benefits of Robert Strandh's implementation
of these, before these get approved for addition to clisp.

The situation is different, however, regarding the compiler: clisp's
compiler does only superficial optimizations (no data flow analysis,
no inlining of local functions).

> * The most useful addition would be to add some form of compiler IR
> from SICL. Robert Strandh has told me that a simple Cleavir-based
> compiler should not be 'too much work'. My understanding is that clisp
> does not do a lot of lisp level optimization, especially type informed
> ones, so that adding the Cleavir compiler would be a huge win as it is
> being used to do all the heavy lisp side optimizations in Clasp.

I agree with all of this.

> The benefits are:
> 1. Having a compiler written in lisp for easier debugging, along with
> the general advantages of lisp.
> 2. Type inferencing
> 3. Loop optimizations
> 4. First-class environments
> 5. Other optimizations

Re 1: clisp's existing compiler is written in Lisp as well. So I don't
see this as a valid argument.

Re 2: Yes, definitely.

Re 3: clisp's LOOP has a number of optimizations already in the macro.
It is possible that cleavir adds more optimizations to it, but this would
not be the major benefit.

Re 4: clisp also has first class environments, I would say. What is the
difference with the cleavir ones?

Re 5: It's here that I see the biggest potential wins: inlining of local
functions, data-flow optimizations, and Scheme-like optimizations.

> SICL is written in such a way where pieces can be selected in a
> modular fashion, so that, for example, once the basic structures are
> in place, pieces can be incrementally added to CLISP. For the purposes
> of GSoC, this means that there would be a guaranteed deliverable, with
> the possibility of some of the above extra pieces if time permits.

This is good; however, we should still strive to define the functionality
of the deliverables. For example, pick the right ones among these:
  - Make the SICL compiler core run in clisp.
  - Connect it to the clisp LAP assembler as backend, so that it can be
    used to create compiled clisp functions.
  - Make the SICL compiler understand elements of the clisp lexical
    environment (i.e. make it possible to use (declare (compile)) in a
    function that is not defined at the top-level).
  - Make the COMPILE function work flawlessly in all cases.
  - Make the DISASSEMBLE function work flawlessly in all cases.
  - Make the COMPILE-FILE function work flawlessly in all cases.
    This implies proper handling of LOAD-TIME-VALUE.

> I have been able to get in touch with the SICL people, with more
> discussoin forthcoming. If there are also willing clisp developers
> available to mentor the clisp side of this project

Yes, I am willing to mentor you with this kind of project.
Sam, are you willing as well?

Bruno

GSoC compiler work, SICL

[Charles Z. <ka...@be...>]

Hi all,

As GSoC applications have opened and the deadline for submissions is
approaching, I'd like to reach out again to the clisp developers.

It seems that the overall evaluation of the feasibility of adding
native file compilation or ahead of time compilation to machine code
is that it is way too ambitious. On the other hand, various
self-contained bugs such as improving local inlining would not be
enough to form a project on its own. However, my main interest is
still compiler work, having had experience writing flow graph
visualizers for sbcl ir1 and other compiler projects.

A good compromise seems to be adding pieces of SICL to clisp, with
Cleavir being the most useful. I'm thinking of tiering the work into
deliverables, each with their own benefits to the CLISP project.

* Probably the easiest piece to add are the various ANSI CL modules,
such as sequence functions, reader, and loop, among others. Of course,
this isn't extremely useful given that CLISP already conforms to ANSI
CL, but there are some novel techniques that Robert Strandh has
introduced that make it worthwhile to look into replacing some of
clisp's implementations.

* The most useful addition would be to add some form of compiler IR
from SICL. Robert Strandh has told me that a simple Cleavir-based
compiler should not be 'too much work'. My understanding is that clisp
does not do a lot of lisp level optimization, especially type informed
ones, so that adding the Cleavir compiler would be a huge win as it is
being used to do all the heavy lisp side optimizations in Clasp. The
benefits are:
1. Having a compiler written in lisp for easier debugging, along with
the general advantages of lisp.
2. Type inferencing
3. Loop optimizations
4. First-class environments
5. Other optimizations
SICL is written in such a way where pieces can be selected in a
modular fashion, so that, for example, once the basic structures are
in place, pieces can be incrementally added to CLISP. For the purposes
of GSoC, this means that there would be a guaranteed deliverable, with
the possibility of some of the above extra pieces if time permits.

I have been able to get in touch with the SICL people, with more
discussoin forthcoming. If there are also willing clisp developers
available to mentor the clisp side of this project, I think this will
be an interesting project that I will draft a proposal along the lines
above. Any feedback much appreciated.

Thanks,
Charles

Re: GSoC Free-Lock Hash Table, Multiple library bindings, etc.

[Bruno H. <br...@cl...>]

Hi Javier,

> 1.- Implement the Free-locking  Hash table using Split Ordered Lists.

I'm not available as a mentor for this topic.

> 2.- implement from 3 to 4 libraries on clisp (Expecting an overhead on
> the numbers Bruno give).

You would have to propose which libraries...

> 3.- Develop a library whit basic functionality of an cloud
> orchestrator  compute node on clisp; bind the libvirt and abstract the
> interfaces on the same fashion of Intel Ciao or Openstack having
> futher development for another GSoC maybe?

This is interesting; especially since you say that you have a background
in this area.

But a Common Lisp binding to libvirt already exists [1], and I would expect
that it does not take long to make this binding actually run in clisp. (Maybe
just 1 week?)

So, to make this a valuable GSoC project, you would need to add significant
functionality on top of it. Just "basic functionality of a cloud orchestrator"
sounds like too little for me.

Can you think about an *interesting* functionality you could build on top
of the libvirt binding? Something that uses the dynamicity of Lisp, or where
a domain-specific language is created? Something that cannot so easily be
done with the Python binding to libvirt, nor with the 'virsh' command?

Bruno

[1] https://www.cliki.net/Lispvirt

GSoC Free-Lock Hash Table, Multiple library bindings, etc.

[Javier A. G. T. <jav...@gm...>]

Hi everyone!
I will like to discuss some ideas for a proposal is it still time for it.
1.- Implement the Free-locking  Hash table using Split Ordered Lists.
2.- implement from 3 to 4 libraries on clisp (Expecting an overhead on
the numbers Bruno give).
3.- Develop a library whit basic functionality of an cloud
orchestrator  compute node on clisp; bind the libvirt and abstract the
interfaces on the same fashion of Intel Ciao or Openstack having
futher development for another GSoC maybe?
Thanks for your time.
Javier

GSoC Presentations.

[Javier A. G. T. <jav...@gm...>]

Hi everyone.
My name is Javier Antonio Gonzalez Trejo from Autonomous University of
Zacatecas and I'd like to work on CLISP for GSoC.
I have done school projects using SBCL and I know the basics of Lisp
workflow so it makes sense that I apply for Entry level tasks.
I have been working on Cloud and I think that will be interesting to
interface the kvm library as well.
 Hopefully is still room on this project regarding GSoC.
Thanks.
Javier

make check failure

[Blake M. <bl...@mc...>]

I am running the latest hg pull on a 64 bit Linux box.  I get the following
on make check:

[...]
Copyright (c) Bruno Haible, Michael Stoll 1992-1993
Copyright (c) Bruno Haible, Marcus Daniels 1994-1997
Copyright (c) Bruno Haible, Pierpaolo Bernardi, Sam Steingold 1998
Copyright (c) Bruno Haible, Sam Steingold 1999-2000
Copyright (c) Sam Steingold, Bruno Haible 2001-2018

Type :h and hit Enter for context help.

;; connecting to
"http://clisp.org/beta/impnotes/id-href.map"...connected...HTTP/1.1
301 Moved Permanently --> "
https://clisp.sourceforge.io/beta/impnotes/id-href.map"
*** - OPEN-HTTP("https://clisp.sourceforge.io/beta/impnotes/id-href.map"):
      HTTPS protocol is not supported yet
Bye.
make: *** [check-doc] Error 1

Re: beta release 2.49.92

[Bruno H. <br...@cl...>]

Blake McBride writes:
> I tried it on 64 bit Linuxmint 17.3
> 
> Build worked
> 
> make check worked
> 
> slime worked
> 
> quick lisp worked

Thanks for the feedback! (I never test farther than "make check".)

Bruno

Re: hg repository changed?

[Jerry J. <log...@gm...>]

On Sat, Mar 3, 2018 at 9:24 AM, Don Cohen
<don...@is...> wrote:
> I've been unable to get updates for several days now:

Sourceforge has been having issues: https://twitter.com/sfnet_ops.
But they said about half an hour ago that services were back online.
Maybe try again.
-- 
Jerry James
http://www.jamezone.org/

hg repository changed?

[<don...@is...>]

I've been unable to get updates for several days now:

  $ hg pull -u
  pulling from http://hg.code.sf.net/p/clisp/clisp
  abort: error: No route to host

Should I be using a different repository now?
Or what?

Re: beta release 2.49.92

[Blake M. <bl...@mc...>]

I tried it on 64 bit Linuxmint 17.3

Build worked

make check worked

slime worked

quick lisp worked

Thanks a lot!!!!

Blake McBride


On Sun, Feb 18, 2018 at 9:46 AM, Bruno Haible <br...@cl...> wrote:

> Hi all,
>
> clisp 2.49.92 is available at
>   https://haible.de/bruno/gnu/clisp-2.49.92.tar.bz2
>
> I invite you all to poke around with it, and report problems, since I'd
> like
> to see a 2.50 release soon that is as robust as possible.
>
> Chages since 2.49.90:
>   - Support for changed process memory map on recent Linux kernels, for
> nearly
>     all architectures.
>   - debug_gcsafety builds should work again.
>   - Fix build failure when building in the srcdir.
>   - Recommend libffcall 2.1 (includes improvements for Linux/arm,
> Linux/mips,
>     Linux/x64_64-x32, Solaris 11, OpenBSD, HardenedBSD).
>
> Features of this beta release (please report issues about it):
>   * Portability - see NEWS file
>   * Many object representations are supported - please use the
>     'multibuild-<platform>' target of Makefile.devel to test all of them.
>   * FFI is supported on all platforms except HP-UX.
>
> Not in the scope of this beta release (no need to report issues on these):
>   * Native Windows ports (mingw and msvc).
>   * Multithreading.
>
> Bruno
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> clisp-devel mailing list
> cli...@li...
> https://lists.sourceforge.net/lists/listinfo/clisp-devel
>

Re: beta release 2.49.92 [Solaris 10/sparc]

[Nemo <cy...@gm...>]

Greetings:

On 18/02/2018, Bruno Haible <br...@cl...> wrote:
> Hi all,
>
> clisp 2.49.92 is available at
>   https://haible.de/bruno/gnu/clisp-2.49.92.tar.bz2
>
> I invite you all to poke around with it, and report problems,

% uname -a
SunOS <host> 5.10 Generic_147147-26 sun4u sparc SUNW,Sun-Blade-2500

CC=/opt/developerstudio12.5/bin/cc \
CXX=/opt/developerstudio12.5/bin/CC \
./configure bld.cc-12.5

"gmake  check-tests" reports:

finished  55 files:               7 errors out of  11,590 tests
  1                   alltest:    0 errors out of     636 tests
  2                     array:    0 errors out of     290 tests
  3                  backquot:    0 errors out of      89 tests
  4                    bin-io:    0 errors out of      15 tests
  5                characters:    0 errors out of     221 tests
  6                      clos:    0 errors out of     496 tests
  7                   defhash:    0 errors out of       6 tests
  8                  encoding:    0 errors out of      36 tests
  9                    eval20:    0 errors out of      50 tests
 10                 ext-clisp:    0 errors out of     115 tests
 11                    floeps:    0 errors out of      20 tests
 12                    format:    0 errors out of     307 tests
 13                 genstream:    0 errors out of      14 tests
 14                      hash:    0 errors out of      48 tests
 15                  hashlong:    0 errors out of      14 tests
 16                 hashtable:    0 errors out of      10 tests
 17                    iofkts:    0 errors out of     228 tests
 18                    lambda:    0 errors out of      90 tests
 19                  lists151:    0 errors out of     201 tests
 20                  lists152:    0 errors out of     255 tests
 21                  lists153:    0 errors out of       1 test
 22                  lists154:    0 errors out of      46 tests
 23                  lists155:    0 errors out of      25 tests
 24                  lists156:    0 errors out of      20 tests
 25                  list-set:    0 errors out of      10 tests
 26                      loop:    0 errors out of     151 tests
 27                    macro8:    0 errors out of     253 tests
 28                       map:    0 errors out of      64 tests
 29                       mop:    0 errors out of     225 tests
 30                    number:    0 errors out of   3,655 tests
 31                   number2:    0 errors out of     331 tests
 32                    pack11:    0 errors out of     211 tests
 33                      path:    0 errors out of     179 tests
 34                 readtable:    0 errors out of      27 tests
 35                      setf:    0 errors out of     197 tests
 36                    socket:    6 errors out of      92 tests
 37                   steele7:    0 errors out of      86 tests
 38                   streams:    0 errors out of     388 tests
 39               streamslong:    0 errors out of      25 tests
 40                   strings:    1 error out of      409 tests
 41                  symbol10:    0 errors out of     152 tests
 42                   symbols:    0 errors out of       6 tests
 43                      time:    0 errors out of      22 tests
 44                     tread:    0 errors out of     395 tests
 45                      type:    0 errors out of     287 tests
 46                unportable:    0 errors out of      31 tests
 47                      weak:    0 errors out of     120 tests
 48                  weakhash:    0 errors out of      26 tests
 49                 weakhash2:    0 errors out of      47 tests
 50                 bind-eval:    0 errors out of      72 tests
 51              bind-compile:    0 errors out of      72 tests
 52                conditions:    0 errors out of      98 tests
 53                  restarts:    0 errors out of      71 tests
 54                  excepsit:    0 errors out of     395 tests
 55                   weakptr:    0 errors out of     260 tests
Real time: 301.96713 sec.
Run time: 296.202 sec.
Space: 1093813408 Bytes
GC: 1518, GC time: 69.85949 sec.

Sincelery,
N.

Re: beta release 2.49.92

[Jean L. <bu...@gn...>]

On Sun, Feb 18, 2018 at 04:46:42PM +0100, Bruno Haible wrote:
> Hi all,
> 
> clisp 2.49.92 is available at
>   https://haible.de/bruno/gnu/clisp-2.49.92.tar.bz2
> 
> I invite you all to poke around with it, and report problems, since I'd like
> to see a 2.50 release soon that is as robust as possible.

Results from my side:

ulimit -s 16384 && ./configure --prefix=/package/prog/clisp-`/bin/date +'''%F-%T-%A'''` --with-module=asdf --with-module=berkeley-db --with-module=bindings/glibc --with-module=clx/new-clx --with-module=dbus --with-module=editor --with-module=fastcgi --with-module=gdbm --with-module=gtk2 --with-module=pcre --with-module=postgresql --with-module=rawsock --with-module=zlib build-`/bin/date +'''%F-%T-%A'''`

finished  56 files:               0 errors out of  11,858 tests
 1                   alltest:    0 errors out of     636 tests
 2                     array:    0 errors out of     290 tests
 3                  backquot:    0 errors out of      89 tests
 4                    bin-io:    0 errors out of      15 tests
 5                characters:    0 errors out of     221 tests
 6                      clos:    0 errors out of     496 tests
 7                   defhash:    0 errors out of       6 tests
 8                  encoding:    0 errors out of      36 tests
 9                    eval20:    0 errors out of      50 tests
10                 ext-clisp:    0 errors out of     118 tests
11                       ffi:    0 errors out of     263 tests
12                    floeps:    0 errors out of      20 tests
13                    format:    0 errors out of     307 tests
14                 genstream:    0 errors out of      14 tests
15                      hash:    0 errors out of      48 tests
16                  hashlong:    0 errors out of      14 tests
17                 hashtable:    0 errors out of      10 tests
18                    iofkts:    0 errors out of     228 tests
19                    lambda:    0 errors out of      90 tests
20                  lists151:    0 errors out of     201 tests
21                  lists152:    0 errors out of     255 tests
22                  lists153:    0 errors out of       1 test
23                  lists154:    0 errors out of      46 tests
24                  lists155:    0 errors out of      25 tests
25                  lists156:    0 errors out of      20 tests
26                  list-set:    0 errors out of      10 tests
27                      loop:    0 errors out of     151 tests
28                    macro8:    0 errors out of     253 tests
29                       map:    0 errors out of      64 tests
30                       mop:    0 errors out of     225 tests
31                    number:    0 errors out of   3,655 tests
32                   number2:    0 errors out of     331 tests
33                    pack11:    0 errors out of     211 tests
34                      path:    0 errors out of     179 tests
35                 readtable:    0 errors out of      27 tests
36                      setf:    0 errors out of     197 tests
37                    socket:    0 errors out of      92 tests
38                   steele7:    0 errors out of      86 tests
39                   streams:    0 errors out of     388 tests
40               streamslong:    0 errors out of      25 tests
41                   strings:    0 errors out of     409 tests
42                  symbol10:    0 errors out of     152 tests
43                   symbols:    0 errors out of       6 tests
44                      time:    0 errors out of      22 tests
45                     tread:    0 errors out of     395 tests
46                      type:    0 errors out of     289 tests
47                unportable:    0 errors out of      31 tests
48                      weak:    0 errors out of     120 tests
49                  weakhash:    0 errors out of      26 tests
50                 weakhash2:    0 errors out of      47 tests
51                 bind-eval:    0 errors out of      72 tests
52              bind-compile:    0 errors out of      72 tests
53                conditions:    0 errors out of      98 tests
54                  restarts:    0 errors out of      71 tests
55                  excepsit:    0 errors out of     395 tests
56                   weakptr:    0 errors out of     260 tests
Real time: 57.23349 sec.
Run time: 43.927937 sec.
Space: 1349468576 Bytes
GC: 1217, GC time: 9.704224 sec.
0
Bye.
(echo *.erg | grep '*' >/dev/null) || (echo "Test failed:" ; ls -l *erg; echo "To see which tests failed, type" ; echo "    cat "`pwd`"/*.erg" ; exit 1)
echo "Test passed."
Test passed.
make[1]: Leaving directory '/media/GNU.Support/sources-other/gnu/clisp-2.49.92/build-2018-02-28-17:53:13-Wednesday/tests'

~~~~
Jean

beta release 2.49.92

[Bruno H. <br...@cl...>]

Hi all,

clisp 2.49.92 is available at
  https://haible.de/bruno/gnu/clisp-2.49.92.tar.bz2

I invite you all to poke around with it, and report problems, since I'd like
to see a 2.50 release soon that is as robust as possible.

Chages since 2.49.90:
  - Support for changed process memory map on recent Linux kernels, for nearly
    all architectures.
  - debug_gcsafety builds should work again.
  - Fix build failure when building in the srcdir.
  - Recommend libffcall 2.1 (includes improvements for Linux/arm, Linux/mips,
    Linux/x64_64-x32, Solaris 11, OpenBSD, HardenedBSD).
  
Features of this beta release (please report issues about it):
  * Portability - see NEWS file
  * Many object representations are supported - please use the
    'multibuild-<platform>' target of Makefile.devel to test all of them.
  * FFI is supported on all platforms except HP-UX.

Not in the scope of this beta release (no need to report issues on these):
  * Native Windows ports (mingw and msvc).
  * Multithreading.

Bruno

Re: new beta release

[Jerry J. <log...@gm...>]

On Sun, Feb 11, 2018 at 6:03 PM, Bruno Haible <br...@cl...> wrote:
> Hi all,
>
> clisp 2.49.90 is available at
>   https://haible.de/bruno/gnu/clisp-2.49.90.tar.bz2
>
> I invite you all to poke around with it, and report problems, since I'd like
> to see a 2.50 release soon that is as robust as possible.

When configured with -cbc, the build fails as follows:

...
Step 2 ===> make SUCCEEDED
Step 3 ===> make check SUCCEEDED
sed: can't read cbcstep4.log: No such file or directory

Perhaps touch should be used to generate empty versions of the log
files that are not generated by running checks.  I will try again with
-cbcx and see how that goes.
-- 
Jerry James
http://www.jamezone.org/

Re: Build failure with clx/new-clx module on Cygwin (was: Build failure with fastcgi module)

[Ken B. <kb...@co...>]

On 3/5/2017 2:40 PM, Bruno Haible wrote:
> Hi Ken,
> 
>> I also tried using angle brackets instead of double quotes around config.h.  But I still got an error:
>>
>> ../../ccmp2c /home/kbrown/src/cygclisp/clisp-2.49-1.20170303hg15769.x86_64/src/clisp/modules/clx/new-clx/clx.f > genclx.c
>> gcc -I/home/kbrown/src/cygclisp/clisp-2.49-1.20170303hg15769.x86_64/src/clisp/src -I/home/kbrown/src/cygclisp/clisp-2.49-1.20170303hg15769.x86_64/build/gllib -I/home/kbrown/src/cygclisp/clisp-2.49-1.20170303hg15769.x86_64/src/clisp/src/gllib -I/home/kbrown/src/cygclisp/clisp-2.49-1.20170303hg15769.x86_64/build/gllib -I/home/kbrown/src/cygclisp/clisp-2.49-1.20170303hg15769.x86_64/src/clisp/src/gllib -I. -ggdb -O2 -pipe -Wimplicit-function-declaration -W -Wswitch -Wcomment -Wpointer-arith -Wreturn-type -Wmissing-declarations -Wimplicit -Wno-sign-compare -Wno-format-nonliteral -O -fno-strict-aliasing -DENABLE_UNICODE -DDYNAMIC_MODULES  -DDLL_EXPORT -DPIC -DWANT_XPM=1 -DWANT_XSHAPE=1 genclx.c -o genclx
>> In file included from /home/kbrown/src/cygclisp/clisp-2.49-1.20170303hg15769.x86_64/build/gllib/stdlib.h:96:0,
>>                   from genclx.c:3:
>> /home/kbrown/src/cygclisp/clisp-2.49-1.20170303hg15769.x86_64/build/gllib/unistd.h:139:3: error: #error "Please include config.h first."
>>    #error "Please include config.h first."
>>     ^
>> In file included from /usr/include/sys/types.h:68:0,
>>                   from /home/kbrown/src/cygclisp/clisp-2.49-1.20170303hg15769.x86_64/build/gllib/sys/types.h:28,
>>                   from /usr/include/stdio.h:61,
>>                   from genclx.c:2:
>>
>>
>> Am I doing something wrong?  There's no "config.h not found" error, so I'm puzzled by the error message.
> 
> It's a constraint imposed by gnulib that config.h needs to be the first
> include file to be included in a compilation unit.
> 
> But more fundamentally, the use of gnulib is fundamentally broken regarding
> clisp modules:
>    - The clx module should not even see build/gllib/unistd.h.
>    - The syscalls module should not rely on the getloadavg module from the
>      clisp core. This is most evidently seen by a link error on AIX.
> So, you cannot do anythere here until this fundamental mistake is fixed.
> clisp's modules are designed as separately compilable units; therefore they
> must use *independent* autoconf configurations; therefore their uses of
> gnulib must be independent as well.

For the record, I found a simple workaround (attached) that allows me to 
build clx/new-clx on Cygwin.

Ken

Re: CLISP GSoC Project Ideas

[Charles Z. <ka...@be...>]

Hi,

I don't mean to interject, but for what it's worth, I know that Clasp,
a 'new' implementation with a C++ substrate, has successfully
integrated Robert Strandh's SICL compiler 'Cleavir' and makes active
use of it. Potentially this makes integrating his compiler frontend
into CLISP more feasible, as there is previous experience of doing the
same with another implementation. At least for me, this would also be
quite interesting, having had some interest and small contributions to
SICL in the past.

Charles

On Sun, Feb 18, 2018 at 6:36 PM, Sam Steingold <sd...@gn...> wrote:
> Hi Charles,
>
> I did not realize that you were writing to clisp-list, not clisp-devel.
> I am replying there, so that Bruno will see it.
> I hope he will volunteer to be the primary mentor.
> I am certainly willing to help too, but Bruno's command of both C and
> CLISP internals is unsurpassed.
>
> Bruno, please respond!
>
> Thanks.
>
>> * Charles Zhang <xneybf@orexryrl.rqh> [2018-02-18 17:57:19 -0800]:
>>
>> Thanks for your encouraging response.
>>
>> After digging deeper into the CLISP internals (reading the byte-code
>> specification), I've decided a do-able and interesting backend project
>> would be to add native file compilation as described here:
>> https://clisp.sourceforge.io/wanted.html
>> My understanding is that this would allow the use of external debuggers
>> like GDB on compiled code and forgo the need for a byte-code interpreter at
>> execution time. In essence, this would be the ahead of time counterpart to
>> the JIT compiler. I do not know which option (to C, LLVM, or GCC IR) would
>> be the easiest to integrate into CLISP, although my intuition tells me that
>> transpilation to C would be easiest. At this point, I am wondering if there
>> is anyone available who is willing to mentor this project, so that I may
>> write a more detailed proposal to be submitted for GSoC and receive general
>> pointers.
>>
>> Charles
>>
>> On Fri, Feb 16, 2018 at 9:52 AM, Sam Steingold <sd...@gn...> wrote:
>>
>>> Hi Charles,
>>>
>>> > * Charles Zhang <xneybf@orexryrl.rqh> [2018-02-16 02:52:26 -0800]:
>>> >
>>> > I'd like to work on GNU CLISP for Google Summer of Code 2018.
>>>
>>> Welcome!
>>>
>>> > Specifically, I am interested in either implementing lock-free
>>> > hash-tables or
>>>
>>> I think getting MT to work has the highest priority ATM.
>>>
>>> > working on the backend, e.g. adding either a bytecode to C transpiler,
>>> > or using bytecode as an IR as a means to compile straight to native
>>> > code, either AOT or adding onto the existing JITC work. Another
>>> > possible idea I'm interested in would be to improve the byte-code
>>> > compiler itself: adding various optimizations not already done
>>>
>>> One compiler improvement that has been on the TODO list as long as I
>>> remember is
>>>
>>> Enhance the compiler so that it can inline local functions.
>>>
>>> (Using local function involves copying the closure object ATM, avoiding
>>> that would be very nice).
>>>
>>> > or (more ambitiously) to perform better type inference. As far as
>>> > I can tell, CLISP does not do as much type-informed optimizations as
>>> > other Common Lisp implmentations.
>>>
>>> I don't think type-informed optimizations would be easy to do in CLISP.
>>>
>>> > I am familiar with the internals of SBCL and know the details of Lisp
>>> > compilation fairly well, so compiler features rather than user-level
>>> > additions like library bindings would be more interesting to work on.
>>> > It would be great to hear thoughts from the CLISP maintainers on what
>>> > seems like a feasible GSoC goal.
>>>
>>> The success of your project depends on your commitment, which depends on
>>> your interest.
>>> Please pick whatever excites you.
>>>
>>> Thanks.
>
> --
> Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1561
> http://steingoldpsychology.com http://www.childpsy.net http://camera.org
> http://islamexposedonline.com http://thereligionofpeace.com
> Would you like a dozen wireless mice to feed the Python book you just bought?



-- 
Class of 2021

CLISP GSoC project ideas

[Bruno H. <br...@cl...>]

Hi Sam,

Before we offer a GSoC project to a specific volunteer, I think we should
collect ideas and see how promising they are.

What are criteria for promising ideas?
  - Don't require too much learning about clisp internals.
  - Be feasible.
  - Require 2-4 months of work.

Ideas I can think of / am aware of:

[from https://clisp.sourceforge.io/wanted.html]
* Remaining work on multithreading: too much learning, too big (2 man-years)
* Multithreaded hash tables: IMO pointless, locking is the way to go
* Octave binding: learning and sizing probably ok, but is it appealing?
* IEEE NaNs and infinities: too small (2 man-weeks)
* Embed clisp: too big (1 man-year)
* Native file compilation: too big (2 man-years)
* Native just-in-time compilation via llvm, libgccjit, or eclipse omr: too big
  (1-2 man-years)
* Compilation to JVM: already done by Clojure, pointless for clisp
* GUI: too big
* SSL bindings: too small (2-4 man-weeks)

[other]
* Source-level debugging with breakpoints in SLIME
  (or is this already done? I don't know)
* Enhance the compiler to do inlining of local functions
  - IMO infeasible without rewriting the compiler frontend to include data
    flow analysis.
* Integrate Robert Strandh's SICL compiler frontend with clisp's bytecode
  backend

Do you have other ideas? Other learning or sizing estimations?

Bruno

Re: __DATE__ & __TIME__

[Bruno H. <br...@cl...>]

Hi Sam,

> 1. I put "#ifdef __DATE__" in constobj.d -- is this really necessary?
> Are there compilers that do not support __DATE__ & __TIME__?

No, __DATE__ and __TIME__ are well supported by all C and C++ compilers.

> 2. https://gcc.gnu.org/onlinedocs/cpp/Standard-Predefined-Macros.html:
> 
> >> If GCC cannot determine the current date, it will emit a warning
> >> message (once per compilation) and __DATE__ will expand to "??? ??
> >> ????".
> 
> Does this really ever happen?

Let's look in the source code. gcc-7.3.0/libcpp/macro.c lines 346..400.
Apparently it can happen in very exotic situations.

The reason __DATE__ and __TIME__ can generate warnings (gcc -Wdate-time)
is because there is a movement to achieve reproducible builds [1]. But
the warning is pretty pointless because the goal of having reproducible builds
can equally be achieved by setting the SOURCE_DATE_EPOCH environment variable
[2]:

$ cat foo.c
#include <stdio.h>
int main ()
{
  printf ("%s %s\n", __DATE__, __TIME__);
}
$ SOURCE_DATE_EPOCH=1600000000 gcc foo.c -Wdate-time
foo.c: In function 'main':
foo.c:4:22: warning: macro "__DATE__" might prevent reproducible builds [-Wdate-time]
   printf ("%s %s\n", __DATE__, __TIME__);
                      ^~~~~~~~
foo.c:4:32: warning: macro "__TIME__" might prevent reproducible builds [-Wdate-time]
   printf ("%s %s\n", __DATE__, __TIME__);
                                ^~~~~~~~
$ ./a.out 
Sep 13 2020 12:26:40

It would be quite some work to make clisp produce reproducible builds (4-8 weeks,
I guess). IMO the work would contain three parts:
  - Remove timestamps from object files.
  - Make the lisp.run invocations during builds not access the current time but instead
    the SOURCE_DATE_EPOCH.
  - Make the memory images immune against ASLR.

Bruno

[1] https://reproducible-builds.org/
[2] https://reproducible-builds.org/specs/source-date-epoch/

__DATE__ & __TIME__

[Sam S. <sd...@gn...>]

Bruno,

1. I put "#ifdef __DATE__" in constobj.d -- is this really necessary?
Are there compilers that do not support __DATE__ & __TIME__?

2. https://gcc.gnu.org/onlinedocs/cpp/Standard-Predefined-Macros.html:

>> If GCC cannot determine the current date, it will emit a warning
>> message (once per compilation) and __DATE__ will expand to "??? ??
>> ????".

Does this really ever happen?
Do we need code to handle this?

Thanks.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1561
http://steingoldpsychology.com http://www.childpsy.net http://memri.org
http://thereligionofpeace.com http://www.memritv.org http://honestreporting.com
A poet who reads his verse in public may have other nasty habits.