AW: thread pools?

[<Joe...@t-...>]

Hi Don,

>Is there some existing implementation of thread pools for clisp?
Not that I've heard of.

I would *believe* the MT authors did not think about pools and that's why there is no mention of them in impnotes.

Conceptually, using or not using pools could be just a *use-thread-pool* choice. However, one should be aware of possible consequences. A handle to a closed socket or closed file should become unusable. But do you remember that ancient bug where CLISP closed the wrong socket fd?
A handle to a "job done" thread should become unusable as well, one would expect IMHO. But if that thread lives on in a pool, wouldn't the handle survive as well? When would you expect FINALIZE to be called?

The only nice solution appears to dissociate the Lisp handle from the OS thread handle. Then, on the Lisp level, FINALIZE and THREAD-INTERRUPT can only ever work on the original thread and no buggy code would be able to send spurious signals to the reused thread.

>Perhaps threads to be used in a pool should always be doing something, like (loop (sleep 1e6))?
That's basically how pools work, except they dispense with a timeout and simply pause/wait forever, i.e. until a signal (job request) kicks in.
Note that expecting THREAD-INTERRUPT to be able to submit a new request to a (hopefully) sleeping thread is a design decision that is at odds with the design I sketched above. Yes, there are different solution paths, with pros and cons.

>Would it be much faster to use an existing thread from a  pool than to create a new one? 
Much? I don't know. However that's typically the reason why they are used and reused.

>Should I write my own?
Only you can tell ;-)
Basically, if THREAD-INTERRUPT works well, you could write your own at Lisp-level, based on the existing primitives. You would have to provide your own THREAD-ACTIVE-P, THREAD-JOIN (perhaps based on EXEMPTION-WAIT?) and possibly more. FINALIZE on a thread object would not be called as early as before.
If you completely wrap all MT:THREAD objects, then you will be able to prevent rogue code from sending interrupts to reused threads, as sketched above. It might be unwieldly, e.g. MT:MUTEX-OWNER returns a thread object; would you wrap that too? Or simply live with the limitations?
Wrap and wrap and wrap and adding another pointer indirection, is that what 70 years of computer science have taught us?

BTW, I see 2 bugs in https://clisp.sourceforge.io/impnotes/mt.html
1. (MT:MUTEX-NAME thread) should operate on a mutex object instead. 
2. (MT:EXEMPTION-NAME thread) should operate on an exemption object instead.

Regards,
	Jörg