Re: [Sbcl-devel] Deadlines trumping WITHOUT-INTERRUPTS

[Tobias C. R. <tc...@fr...>]

Nikodemus Siivola <nik...@ra...> writes:

> Gabor wrote:
>
> > I'd vote for with-interrupts affecting deadlines and existing code be audited.
>
> An audit does cover my concern I guess, along with an Incompatible
> Change note in NEWS.

An audit along what policy? I came to the realization that deadlines
should be _signaled_ at places not only safe to unwind, but as reentrant
as possible, too. In particular, this means deadlines should be signaled
outside of critical sections. 


[To be precise: user's handler should be run outside of critical
sections; the deadlines may be signaled no matter where as long as the
signal is unwinded to safe place e.g. via a facility like your
HANDLER-UNWIND idea.]


So as a policy, deadlines handlers established via HANDLER-BIND should
never be executed holding on a lock for two reasons:

  a) if a user's handler is executed with implicitly holding on an
     internal lock means that the handler is blocking other threads from
     doing something useful -- not to mention that it's prone to
     deadlock.

  b) in particular, the handler cannot call functions operating on the
     same lock except if those operations use WITH-RECURSIVE-LOCK.

     That's the wake-up-myself-from-deadline-handler issue.


> Right, eugh. I knew this had to be harder than it looked like.
>
> Sketch 2: hide handlers established outside the HANDLER-UNWIND from
> inside it. Inner handlers work normally, as they are presumably aware
> of the critical section. If no inner handler is found, unwind from the
> critical section and look for outer handlers.
>
> (defvar *handler-unwind* nil)
>
> (defmacro handler-unwind (&body body)
>   `(flet ((call-with-unwind ()
>             (progn ,@body)))
>      (let ((cont (cons nil nil)))
>        (declare (dynamic-extent cont))
>        (block 'handler-unwind
>          (tagbody
>             (dx-flet ((unwind (fun args)
>                               (setf (car cont) fun
>                                     (cdr cont) args)
>                               (go :unwind)))
>               (let ((*handler-unwind* #'unwind)
>                     (*handler-clusters* nil))
>                 (return-from handler-unwind (call-with-unwind))))
>           :unwind
>             (apply (car cont) (cdr cont)))))))
>
> ;;; SIGNAL is just like now: outer handlers are completely invisible to it.

That's not enough for a policy like above. For that, it seems like it
has to boil down to a HANDLER-CASE on SERIOUS-CONDITION (e.g. along your
proposed modification of WAIT-ON-SEMAPHORE you showed me a few days
ago.) I'm not sure how expensive that is.

Another bit:

There are lots of places that look like

  (tagbody
     (without-interrupts
       ...
       (when (foop) (:go :error-foo)))

   :error-foo
     ;; make sure not to signal inside W/O-INTERRUPTs.
     (error "FOO!"

fd-stream.lisp is full of that and more abominations.

Your first implementation of HANDLER-UNWIND could have beeen applied to
those cases, as well. But the second one can't.

  -T.