clisp-devel

Re: CLISP on Haiku

[Bruno H. <br...@cl...>]

Alexandru Popa wrote:
> It turns out the problem of removing :HAIKU from *features* when (require
> "asdf") is a known bug, which was already solved:

Indeed, this commit appears to be the fix:
https://gitlab.common-lisp.net/asdf/asdf/-/commit/2e36695803f46a88b3d6f7b1e1c1e980b12bf720

> after loading ASDF,
> *features* contain :HAIKU, :UNIX, OS-HAIKU and :OS-UNIX. However, both
> CLISP and QuickLisp use an older version which does not include the fix.
> 
> Can ASDF version shipped with CLISP be updated (of course, I will also
> contact QuickLisp for this as well)?

Good idea. Done: CLISP now ships version 3.3.6.7. I'll let you deal with
the QuickLisp people.

> One more request from ASDF team is to provide a Docker image with Haiku +
> CLISP for testing purposes. Is it possible?

A Docker image? No, this is not easily possible. Haiku is an OS.

Yes, I know it is possible to use FreeBSD, NetBSD, OpenBSD in GitHub CI;
these are then virtual machines running inside a Linux machine. But I'm
not aware of the same thing for Haiku.

Bruno

Re: CLISP on Haiku

[Alexandru P. <al...@gm...>]

Hi Bruno.


> Good idea. Done: CLISP now ships version 3.3.6.7. I'll let you deal with
> the QuickLisp people.


Thank you.


> A Docker image? No, this is not easily possible. Haiku is an OS.
>
> Yes, I know it is possible to use FreeBSD, NetBSD, OpenBSD in GitHub CI;
> these are then virtual machines running inside a Linux machine. But I'm
> not aware of the same thing for Haiku.
>

One example of Haiku Docker image (without CLIPS installed) is
https://github.com/hectorm/docker-qemu-haiku, so I think it is possible.
ASDF team can help setting up such an image for CLIPS. Robert Goldman (
rpg...@si..., at asdf-devel mailing list) from ASDF team wrote:
```
As I said, anything that the CLISP team could do to support ASDF testing
would be welcome. I am *somewhat* knowledgeable about the CL Foundation
Docker images, and have managed to roll new ones for SBCL, Allegro, etc.
for testing purposes, so I could help with Dockerizing CLISP if there's
anyone who's interested.
'''

Alexandru

Re: CLISP on Haiku

[Bruno H. <br...@cl...>]

Hi Alexandru,

Alexandru Popa wrote:
> > A Docker image? No, this is not easily possible. Haiku is an OS.
> 
> One example of Haiku Docker image (without CLISP installed) is
> https://github.com/hectorm/docker-qemu-haiku

Oh, I see. Indeed, Haiku can then be handled like the *BSDs or Solaris 11
regarding CI testing.

> so I think it is possible.
> ASDF team can help setting up such an image for CLISP. Robert Goldman (
> rpg...@si..., at asdf-devel mailing list) from ASDF team wrote:
> ```
> As I said, anything that the CLISP team could do to support ASDF testing
> would be welcome. I am *somewhat* knowledgeable about the CL Foundation
> Docker images, and have managed to roll new ones for SBCL, Allegro, etc.
> for testing purposes, so I could help with Dockerizing CLISP if there's
> anyone who's interested.
> '''

With the Haiku Docker image that you cited, the ASDF team can build their
CI for CLISP on Haiku (and for SBCL on Haiku, if they want to).

I think all you need to provide to Robert Goldman are
  - The package name that he needs to pass in the
    "vmshell pkgman install -y ..."
    command.
  - The command to run CLISP (is it 'clisp'? 'clisp -Kbase'? 'clisp -Kfull'?).

This way, Haiku will be loaded from a docker image, and the clisp binaries
from the Haiku package repositories.

Thus there's no need for a Haiku+CLISP docker image; that would only be
needed if downloading from the Haiku package repositories were unbearably
slow or too unreliable.

Bruno

Re: CLISP on Haiku

[Alexandru P. <al...@gm...>]

Hi Bruno,

Recently I worked on porting SBCL to Haiku, mostly by upstreaming already
existing patches. One of them is related to SB-BSD-SOCKET contrib module.
Now, I would like to implement Haiku specific fixes also in CLISP module
ROWSOCK.

The fix consists of the following observations:
- Haiku does not define ESOCKTNOSUPPORT error and never signals it,
- Haiku seems to potentially signal EAFNOSUPPORT error when socket fails to
be created.

Attached is SBCL patch (not for CLISP). Can you guide me to prepare the fix
for CLISP / Haiku?

Thank you,
Alexandru

Re: CLISP on Haiku

[Bruno H. <br...@cl...>]

Hi Alexandru,

> The fix consists of the following observations:
> - Haiku does not define ESOCKTNOSUPPORT error and never signals it,
> - Haiku seems to potentially signal EAFNOSUPPORT error when socket fails to
> be created.
> 
> Attached is SBCL patch (not for CLISP). Can you guide me to prepare the fix
> for CLISP / Haiku?

I think it is better if you report these bugs in the Haiku bug tracker [1],
ideally with a sample C program for each. Then there is a chance that they
get fixed, and then programs like SBCL and CLISP, but also many others, don't
have to use special-case code for Haiku.

Bruno

[1] https://dev.haiku-os.org/

Re: CLISP on Haiku

[Alexandru P. <al...@gm...>]

Hi Bruno,

Haiku does not consider this a bug. There is nothing to "solve" in Haiku.
Different OSes behave differently. Haiku instead of some error uses a more
general one. The OS specific patches are available in almost all OSes:
*BSD, Solaris / Indiana, Windows (of course) etc.

I would prepare the patch myself, jest let me know where to start. What
file(s) to consider. I found the definition of function
RAWSOCK:MAKE-SOCKADDR in modules/rawsock/rawsock.c in lines 255-279. This
file seems to be C source, which resembles Common Lisp structure (is
&optional somehow handled in C?). I just cannot find any error definitions
and handling, maybe this is up to underlying OS 3rd party libraries.

Can you guide me (probably by pointing to relevant documentation) in the
following:
- What is the architecture of CLISP modules? I see many of them use some
glue C code. What is its purpose? How it is used? I see the tool
clisp-link, which seems to be related to modules loading (or
installation?). How it is used?
- Which existing rawsock tests can I use? I see rawsock/test.tst seems to
be Common Lisp code. Does its loading starts some tests or just defines
some specific functions for testing? I also see
modules/rawsock/demos/sniffer.lisp. Can it be used for debugging / testing?

I think it is better if you report these bugs in the Haiku bug tracker [1],
> ideally with a sample C program for each. Then there is a chance that they
> get fixed, and then programs like SBCL and CLISP, but also many others,
> don't
> have to use special-case code for Haiku.
>
> Bruno
>
> [1] https://dev.haiku-os.org/
>

Re: CLISP on Haiku

[Bruno H. <br...@cl...>]

Alexandru Popa wrote:
> Haiku does not consider this a bug. There is nothing to "solve" in Haiku.

I don't know where you got this information from. Socket behaviour is
generally specified by POSIX, and the Haiku bug tracker has a category
"System/POSIX".

To show you how to report deviations from POSIX, I entered this one:
https://dev.haiku-os.org/ticket/19347

> Different OSes behave differently. Haiku instead of some error uses a more
> general one.

This is likely a bug w.r.t. POSIX as well: POSIX specifes which error code
is used in which case. Search for ESOCKTNOSUPPORT in
https://pubs.opengroup.org/onlinepubs/9799919799/functions/V2_chap02.html
https://pubs.opengroup.org/onlinepubs/9799919799/functions/socket.html
If you find that Haiku does not behave like this (it most likely doesn't,
since it does not even have the definition of ESOCKTNOSUPPORT), please report
a bug.

> The OS specific patches are available in almost all OSes:
> *BSD, Solaris / Indiana, Windows (of course) etc.

But we want to have as few conditionals in the code as possible. While
it is unlikely that bug reports from us could change the mind of the *BSD
people (sockets were first defined by BSD, after all), we should do that
w.r.t. Haiku.

> I would prepare the patch myself, jest let me know where to start. What
> file(s) to consider.

CLISP has two socket facilities: a higher-level one in src/socket.d, and a
lower-level on in module rawsock. Which one you need to fix for e.g. SLIME
to work, I cannot tell. Note that src/socket.d does NOT use rawsock; rawsock
really is separate.

> I found the definition of function
> RAWSOCK:MAKE-SOCKADDR in modules/rawsock/rawsock.c in lines 255-279. This
> file seems to be C source, which resembles Common Lisp structure (is
> &optional somehow handled in C?).

Yes. &optional is handled through the missingp() macro.

> I just cannot find any error definitions
> and handling, maybe this is up to underlying OS 3rd party libraries.

The 'define-condition' invocations at the end of rawsock/sock.lisp ?

> Can you guide me (probably by pointing to relevant documentation) in the
> following:
> - What is the architecture of CLISP modules?

See https://clisp.sourceforge.io/impnotes.html#modules

> I see many of them use some
> glue C code. What is its purpose? How it is used?

See https://clisp.sourceforge.io/impnotes.html#modprep

> I see the tool
> clisp-link, which seems to be related to modules loading (or
> installation?). How it is used?

See https://clisp.sourceforge.io/clisp-link.html

> - Which existing rawsock tests can I use? I see rawsock/test.tst seems to
> be Common Lisp code. Does its loading starts some tests or just defines
> some specific functions for testing?

I don't know. Generally, my advice would be to try things on Linux or FreeBSD
first, and then only see whether Haiku behaves differently.

> I also see
> modules/rawsock/demos/sniffer.lisp. Can it be used for debugging / testing?

It's marked as a demo. So it might be useful for limited-scope testing. Maybe.

Bruno

Re: CLISP on Haiku

[Alexandru P. <al...@gm...>]

Thank you Bruno for documentation links and especially for the bug report.

I don't know where you got this information from. Socket behaviour is
> generally specified by POSIX, and the Haiku bug tracker has a category
> "System/POSIX".
>
I understand this from the following response of Haiku OS developer:
https://discuss.haiku-os.org/t/make-ansi-common-lisp-available-on-haiku-again/15780/135
.


To show you how to report deviations from POSIX, I entered this one:
> https://dev.haiku-os.org/ticket/19347

Thank you, I reported several bugs here in the past.

This is likely a bug w.r.t. POSIX as well: POSIX specifes which error code
> is used in which case. Search for ESOCKTNOSUPPORT in
> https://pubs.opengroup.org/onlinepubs/9799919799/functions/V2_chap02.html
> https://pubs.opengroup.org/onlinepubs/9799919799/functions/socket.html
> If you find that Haiku does not behave like this (it most likely doesn't,
> since it does not even have the definition of ESOCKTNOSUPPORT), please
> report
> a bug.
>
Added this information to your ticket.

For now, I want to concentrate on failing tests:
-rw-r--r-- 1 user root 4444 окт.  22 19:02 socket.erg
-rw-r--r-- 1 user root  769 окт.  22 19:02 streams.erg
-rw-r--r-- 1 user root  528 окт.  22 19:02 streamslong.erg

Re: CLISP on Haiku

[Bruno H. <br...@cl...>]

Alexandru Popa wrote:
> I don't know where you got this information from. Socket behaviour is
> > generally specified by POSIX, and the Haiku bug tracker has a category
> > "System/POSIX".
> >
> I understand this from the following response of Haiku OS developer:
> https://discuss.haiku-os.org/t/make-ansi-common-lisp-available-on-haiku-again/15780/135

That was the answer from someone who had not looked at what POSIX says.

> > This is likely a bug w.r.t. POSIX as well: POSIX specifes which error code
> > is used in which case. Search for ESOCKTNOSUPPORT in
> > https://pubs.opengroup.org/onlinepubs/9799919799/functions/V2_chap02.html
> > https://pubs.opengroup.org/onlinepubs/9799919799/functions/socket.html
> > If you find that Haiku does not behave like this (it most likely doesn't,
> > since it does not even have the definition of ESOCKTNOSUPPORT), please
> > report
> > a bug.
> >
> Added this information to your ticket.

Different issues should be stated in separate tickets. When a ticket (like
this one) is already closed, waddlesplash will not keep it on his TODO list.

> For now, I want to concentrate on failing tests:
> -rw-r--r-- 1 user root  769 окт.  22 19:02 streams.erg
> -rw-r--r-- 1 user root  528 окт.  22 19:02 streamslong.erg

I think I fixed these already on 2024-11-03, no?

> -rw-r--r-- 1 user root 4444 окт.  22 19:02 socket.erg

For these test failures, only socket.d is relevant; the module 'rawsock'
is irrelevant. 

Bruno

Re: CLISP on Haiku

[Alexandru P. <al...@gm...>]

Hi Bruno,


> Different issues should be stated in separate tickets. When a ticket (like
> this one) is already closed, waddlesplash will not keep it on his TODO
> list.
>
It seems waddlesplash indeed has this in his TODO list.


> > For now, I want to concentrate on failing tests:
> > -rw-r--r-- 1 user root  769 окт.  22 19:02 streams.erg
> > -rw-r--r-- 1 user root  528 окт.  22 19:02 streamslong.erg
>
> I think I fixed these already on 2024-11-03, no?
>
Ah, you are right. I was looking at older logs.


> > -rw-r--r-- 1 user root 4444 окт.  22 19:02 socket.erg
>
> For these test failures, only socket.d is relevant; the module 'rawsock'
> is irrelevant.
>
Thank you for information. I will take a look at src/socket.d, but if CLISP
has the same problem with sockets as SBCL, it will be solved in Haiku.

Alexandru

Re: CLISP on Haiku

[Alexandru P. <al...@gm...>]

Hi Bruno,

I freshly build and retest (make check) CLISP on Haiku. Two .erg files are
generated in tests folder: socket.erg and weakhash.erg (this one is empty).
The socket.erg content is the following:

Form: (CHECK-OS-ERROR (SOCKET-CONNECT 12345 "localhost" :BUFFERED NIL
:TIMEOUT 0) (:ECONNREFUSED -2147454944))
CORRECT: T
CLISP  : -2147483639
OUT:
"[OS-ERROR]: OS-ERROR(-2147483639): Operation timed out
"

This fail does not seem to be related to missing from Haiku ESOCKTNOSUPPORT
error code. I analyzed tests/socket.tst file and found this operation fails
similarly on Windows and CYGWIN. So I added the exception for Haiku.
Changed from:

(check-os-error (socket:socket-connect 12345 "localhost" :buffered nil
:timeout 0)
  #.(let ((os (ext:operating-system-type)))
      (cond ((equal os "Windows") '(:ETIMEDOUT 10060))
            ((equal os "CYGWIN") `(:ETIMEDOUT ,+ETIMEDOUT+))
            (t `(:ECONNREFUSED ,+ECONNREFUSED+)))))
T

to:

(check-os-error (socket:socket-connect 12345 "localhost" :buffered nil
:timeout 0)
  #.(let ((os (ext:operating-system-type)))
      (cond ((equal os "Windows") '(:ETIMEDOUT 10060))
            ((equal os "CYGWIN") `(:ETIMEDOUT ,+ETIMEDOUT+))
            ((equal os "Haiku) `(:ETIMEDOUT ,+ETIMEDOUT+))
            (t `(:ECONNREFUSED ,+ECONNREFUSED+)))))
T

I did not modify tests for:
(socket:socket-connect 12345 "localhost" :timeout 0)
(socket:socket-connect 12345 "localhost" :timeout 30)

After this change, make check still fails, no weakhash.erg is generated and
socket.erg is generated but is empty. The last log before test ending is:

*** - READ: input stream
      #<INPUT BUFFERED FILE-STREAM CHARACTER #P"socket.tst" @730> ends
within
      a string

The file socket.tst has 729 lines.

I am not sure how to interpret this result. Also, I am not the proper
person to judge if this change can be considered as "fix" or as "hiding a
problem". Just report it to you. Maybe CLISP sockets support on Haiku are
in better shape than it may seem.

Also, the tests with comment:

;; This test frequently fails on Haiku.

do not seem to fail on Haiku.

Alexandru