Steel Bank Common Lisp

On Fri, May 27, 2005 at 12:35:58PM +0100, Gustavo Felisberto wrote:
> http://www.sbcl.org/manual/Threading.html#Threading
> 
> that in itself is a bit contradictory as it says it needs a NPTL system
> and then says:
> "On Linux x86, threading is implemented using |clone()| and does not
> involve pthreads. This is not because there is anything wrong with

This is not contradictory at all.  It requires NPTL because pre-NPTL
systems don't (usually) have futex support - and it's more likely
that people know whether they have NPTL than whether they have a
kernel that supports some syscall they've probably never heard of.

> pthreads /per se/, but there is plenty wrong (from our perspective) with
> LinuxThreads. SBCL threads are mapped 1:1 onto Linux tasks which share a
> VM but nothing else - each has its own process id and can be seen in
> e.g. |ps| output."
> in
> http://www.sbcl.org/manual/Implementation--Linux-x86-.html#Implementation%20(Linux%20x86)
> 
> which is not true.

What's not true?

* (sb-thread:make-thread (lambda () (sleep 300)))

12554
* (sb-thread:make-thread (lambda () (sleep 300)))

12555
* (sb-thread:make-thread (lambda () (sleep 300)))

12556
* (asdf:run-shell-command "ps axf | grep sbcl >/dev/tty")
12553 pts/1    S+     0:01              |       \_ sbcl
12554 pts/1    S+     0:00              |           \_ sbcl
12555 pts/1    S+     0:00              |           \_ sbcl
12556 pts/1    S+     0:00              |           \_ sbcl
12560 pts/1    R      0:00              |           \_ /bin/sh -c ps axf | grep sbcl >/dev/tty

(OK, technically I think they may be sharing the file descriptor table
as well as the VM, but they're not calling pthreads functions and they
definitely have distinct pids)


-dan

When i has reading the docs i saw "On Linux x86, threading is
implemented using clone()" this is refering to "On Linux x86, sbcl is
using threads using clone()" When i read that i understood that the docs
said that linux used clone for threads (as in the old pre nptl).

It was my mistake, i should have read with more attention. I'm sory if i
caused so much commotion.

-- 
Gustavo Felisberto
(HumpBack)
Web: http://dev.gentoo.org/~humpback
Blog: http://blog.felisberto.net/
------------
It's most certainly GNU/Linux, not Linux. Read more at
http://www.gnu.org/gnu/why-gnu-linux.html .
-------------

On Fri, May 27, 2005 at 01:19:06PM +0100, Gustavo Felisberto wrote:
> When i has reading the docs i saw "On Linux x86, threading is
> implemented using clone()" this is refering to "On Linux x86, sbcl is
> using threads using clone()" When i read that i understood that the docs
> said that linux used clone for threads (as in the old pre nptl).

You seem still to be confused.  For the record:

- there are two threading libraries for Linux: LinuxThreads (the old one)
  and NPTL (the new one)

- They both provide a POSIX Threads ("pthreads") interface, and are both
  implemented with use of clone()

- however, they typically use different options to clone(); a LinuxThreads
  system will create a separate pid for each thread, whereas a system
  with NPTL libraries will share the pid between threads.  

- SBCL does not currently[*] use the pthreads interface at all, because when
  the threading support was added NPTL was either in early development or 
  did not yet exist, and LinuxThreads is badly broken.  Instead it calls
  clone() directly.  The options to clone() that it uses cause it to
  create a separate pid for each thread

- However, it does require kernel support for futexes (for its mutex 
  implementation).  As not many people knwo whether they've got this,
  and as it's available on pretty much exactly the same systems as 
  NPTL (it is part of the kernel support required for NPTL), we describe
  SBCL as needing NPTL



[*] There's a currently-stalled branch of SBCL in CVS which uses the pthread
interfaces, but it's stuck on an interaction between POSIX 1003.1b semaphores
and copying GC, which I cannot see how to resolve in a non-icky way.  See
http://ww.telent.net/diary/2005/3/#6.793

On Fri, 27 May 2005 13:54:50 +0100, <dan@...> wrote:
>
> - However, it does require kernel support for futexes (for its mutex 
>   implementation).  As not many people knwo whether they've got this,
>   and as it's available on pretty much exactly the same systems as 
>   NPTL (it is part of the kernel support required for NPTL), we describe
>   SBCL as needing NPTL

This is a poor approach, it would be better to state:

... with NPTL kernel support, (more specifically futex support), ...

Once a linux system has been converted to NPTL, it is stuck in 2.6
kernel land.  There are many products, for instance VMware that do not
support 2.6 kernels. With NPTL in place, I cannot drop back into 2.4
to provide a test case to VMware.  NPTL requires more than a kernel
change, if I recall correctly, I had to rebuild libc and a few other
tools.  A futex changes sounds like I have the opportunity to return
to 2.4 kernels to deal with support issues.


-- 
With sufficient thrust, pigs fly fine.

GP lisper <spambait@...> writes:

> This is a poor approach, it would be better to state:
>
> ... with NPTL kernel support, (more specifically futex support), ...

This may be a fair point: I suppose we could say something more like

| This requires x86 and Linux kernel 2.6 or systems with NPTL backports.

or

| Queues require the sys_futex() system call to be available: this is
| the reason for the NPTL requirement. We test at runtime that this
| system call exists.

Oh, but look!  WE ALREADY DO!  

> Once a linux system has been converted to NPTL, it is stuck in 2.6
> kernel land.  There are many products, for instance VMware that do not
> support 2.6 kernels. 

The VMWare web site lists several 2.6-series kernels that it works
with: <http://www.vmware.com/support/ws5/doc/intro_hostreq_ws.html>.
I don't know what problems you're experiencing, but you should probably
take them up with vmware support.



-dan

On Sat, 28 May 2005 02:28:07 +0100, <dan@...> wrote:
> GP lisper <spambait@...> writes:
>
>| Queues require the sys_futex() system call to be available: this is
>| the reason for the NPTL requirement. We test at runtime that this
>| system call exists.
>
> Oh, but look!  WE ALREADY DO!  

Do what Dan?  Previously you have stated:

This is not contradictory at all.  It requires NPTL because pre-NPTL
systems don't (usually) have futex support - and it's more likely
that people know whether they have NPTL than whether they have a
kernel that supports some syscall they've probably never heard of.

and

- However, it does require kernel support for futexes (for its mutex 
  implementation).  As not many people knwo whether they've got this,
  and as it's available on pretty much exactly the same systems as 
  NPTL (it is part of the kernel support required for NPTL), we describe
  SBCL as needing NPTL


So which is it, just futex (i.e. kernel only) support or futex plus
NPTL (additional libs) ?

For that matter, I don't even see an xconfig setting for futex.
CONFIG_FUTEX=y appears in General Setup of .config, I know it I've
toggled it in the past (x86).


> The VMWare web site lists several 2.6-series kernels that it works

Oh good, thanks, my friend that works for VMware didn't mention it at
lunch.  Odds are it's the broken kernels, but at least they moved
forward.  2.6 wouldn't support full screen here on multiple displays.
Too bad it wasn't months ago when I first needed it.

-- 
With sufficient thrust, pigs fly fine.