Re: [TCLCORE] Coroutines in HEAD

[Lars H. <Lar...@re...>]

miguel skrev:
> I managed to commit an experimental implementation of coroutines in time for 
> 8.6a2. This provides two new commands
>    ::tcl::unsupported::coroutine
>    ::tcl::unsupported::yield
> 
> A brief description can be found at http://msofer.com:8080/wiki?name=Coroutines, 
> otherwise the code in tests/unsupported.test is the only documentation.
> 
> Test! Enjoy! Help make it better

This might fall in the "help make better" category, although it's 
really some loose sketches for an alternative realization of the same 
concept that I've been contemplating on and off during the last couple 
of years (judging from http://wiki.tcl.tk/13232, since April 2005). I'm 
posting this mostly to provide an alternative point of view -- maybe 
some details of the Tcl interface would be better if done differently? 
-- although it seems this approach could actually do some things 
mentioned as unsolved in the NRE implementation (moving coroutines 
between threads, handle C-coded commands calling Tcl_Eval).


BASIC PREMISE

Primarily I wanted a non-nesting [vwait] -- one which would not hang in

after 1000 {
    after 1000 {set ::a 1; vwait ::b; puts "Finished"}
    vwait ::a
    set ::b 1
}

This meant the [vwait] would somehow have to save away what was 
currently in the C call stack, drop out into the outer event loop and 
process events there for a while, until the variable has been set, at 
which point everything would have to be put back so that processing 
could continue.

Also, this would have to be accomplished "within the box" -- no fancy C 
compiler or OS features could be relied on, and recursive calls should 
still be supported.


SOLUTION

Since there's no safe way to operate on the C stack from outside, the 
only way to get it out of the way is to explicitly unwind it -- make 
every C function in there return to its caller. That's not too 
different from having an error propagate down the call stack, so:

   Yielding is done by returning with a new [return] code,
   TCL_SUSPEND, say.

   When a yielding unwind is in progress, the interpreter result
   is a list. Each entity in a call chain receiving a TCL_SUSPEND
   result code from a call it made must do one of the following:

   1. Append to the interp result a list element with sufficient
      information that it can later reconstruct the exact state
      it was in immediately before receiving the TCL_SUSPEND,
      and thus resume processing from where it was suspended.
      Then return to caller with TCL_SUSPEND code.

   2. Throw an error "Cannot be suspended".

   3. Catch the yield, if the entity is prepared to handle it.

Normally, a yield would proceed all the way down to the main event 
loop, where it gets handed off to [bgerror] (or the like), and that 
command is then responsible for arranging so that the suspended calls 
are resumed at a suitable point in the future.

A tricky detail is that there would now have to be two entry points for 
the C implementation of each Tcl command: one which is used when 
calling it normally, and one which is used when resuming it. I'll 
return to that matter below, but for now, let's look at what the script 
level behaviour is supposed to be.


TCL API

I imagined the Tcl side of this could be handled with only two commands 
(although [catch] and [return] would probably figure as prominently as 
with any new control structure):

   suspend ?$arg ...?
   resume $unwinding ?-code $code? ?-level $level? ?...? ?$result?

[suspend] stops execution at some point and causes the call stack to 
unwind, whereas [resume] resumes it. If you do

   proc suspending {args} {list [suspend {*}$args] "in suspending"}
   catch {suspending foo bar} res

then the result will be TCL_SUSPEND and $res will be an unwinding that 
contains the arguments of the [suspend] in its 0th element, i.e., it 
looks like

   {foo bar} {...

where the ... denotes the beginning of the serialization of the 
internal state of the call stack entity corresponding to the 
[suspending] procedure (yes, that is potentially quite a mouthful).

The [resume] arguments following the $unwinding are supposed to be the 
same as for [return], but control how the original [suspend] behaves 
when resumed. Thus if one then does

   resume $res "Return from with"

then the result will be the list

   {Return from with} {in suspending}

because [resume] put back the [suspending] proc and the [suspend] 
command, the latter returns with TCL_OK (since there was no other -code 
specified), the [list] in suspended is called as

   list {Return from with} "in suspending"

and the result of that becomes the result of [suspending] as a whole, 
which is then also the result of the [resume].


C API