Re: exemption-wait question

[Vladimir T. <vtz...@gm...>]

On 11/1/10, Don Cohen <don...@is...> wrote:
>  > If thread is interrupted while in exemption-wait the interrupt
>  > function will be executed with mutex held.
> So if you interrupt the thread to go into the debugger, then all other
> threads that need to get the mutex (like to return write access) are
> stuck while you debug.  And if you then interrupt another to go into
> the debugger you can't even get into the debugger cause you're stuck
> waiting for the mutex before entering the debugger.
> It seems to me that if you interrupt exemption-wait you should not
> wait to get the lock.  That's supposed to happen only before you exit
> exemption-wait and you should be able to interrupt and go into the
> debugger before that.

If you are brave enough - unlock the mutex from interrupt function -
but be careful to lock it again on exit.
btw: debugging in presence of many threads is quite adventurous. I
find it mostly useful for fixing deadlocks - for everything else I
prefer logging.

Let's explain why it works this way:
Suppose you are stuck in pthread_cond_wait() and you want to interrupt
it. According to POSIX standard this function never returns EINTR (if
it gets signal). Rather it calls signal handler and continues to wait
as if nothing happened. Within signal handler no lisp code can be
executed (and actually very limited set of C functions are allowed) .
The only way to implement thread-interrupt while we are waiting in
pthread_cond_wait() is to signal the condition/exemption, run the
interrupt function and continue to wait. After we signal the exemption
- mutex is reacquired automatically by the OS. As final result - lock
is held when the interrupt functions is executed.

>  > On non-local exit from the function - all unwind-protect forms will
>  > be executed and mutex in the above example will be unlocked. In
>  > case of normal exit from interrupt function - exemption-wait will
>  > continue to wait.
> "The function" means the one running the code above?
"interrupt function" means the function passed as :function to
thread-interrupt to be executed in the context of the thread.

>  > With native OS preemptive threads - we do not have control on the
>  > scheduler. We may influence it with mutex and exemptions.
> This seems strange.  Wouldn't one normally wish to control which
> threads run when more are ready to run than there are processors
> available to run them?  I'm not blaming you, of course!
> (Unless you have a lot more to do with the design than I imagine.)
Not sure I understand. You control threads being executed via mutexes
and exemptions - that's all.

>  > > BTW, it seems to me that there should also be some some way to control
>  > > which of several threads waiting to run should be scheduled next.  Why
>  > > is that either not possible or a bad idea?
>  > Use distinct exemptions for this (and share the mutex if needed).
> Just off hand I don't see how this solves the problem.
> Suppose each thread had a priority (integer) - how would you arrange
> to always resume a thread of highest priority among those ready to
> run?
What do you mean by "thread ready to run"? All threads are running all
the time until they exit - just some of them are blocked in calls like
mutex-lock, exemption-wait, read(), etc and do not consume cpu cycles.
Only OS kernel knows when a thread is in such blocked state and why.

Looks like you want to process some prioritized tasks. If so - I would
do something like:
1. define a job
2. maintain sorted by priority collection of jobs. guard with mutex
and signal an exemption when new highest priority item arrives.
3. create (pool of) thread(s) - for jobs execution. wait with
exemption on the collection of jobs and execute when available (when
collection is not empty).
4. In case all threads are busy executing jobs and new highest
priority job arrives and you are absolutely sure you want to execute
it immediately - spawn a new thread or implement kind of job
cancellation in order to release thread from the pool.