Re: [TCLCORE] Coroutines in HEAD

[Lars H. <Lar...@re...>]

Neil Madden skrev:
> 
> On 24 Aug 2008, at 21:02, Lars Hellström wrote:
>>
>> *Existence.* Obviously, a big difference is that an implementation of
>> [coroutine]/[yield] exists, whereas [suspend]/[resume] is at best a 
>> sketch.
>>
>> *Model.* [coroutine]/[yield] is, as far as I can tell, modelled after
>> the concept of a generator: a subroutine than can return several times.
> 
> Similar, yes, although generators are not usually first-class objects.

Did that come out backwards? The NRE coroutines doesn't seem to be 
first-class objects, but I can imagine coroutines elsewhere as a rule 
being first-class.

>> [suspend]/[resume] rather follows the continuation model: the rough
>> idea of "the program from this point on" is given tangible existence.
>> It is different from the LISPish (call-cc) in that the continuation is
>> not given the form of an anonymous function, but that's natural given
>> the more imperative nature of Tcl, and it is also different in that it
>> is not the part of the program that calls [suspend] which gets to
>> handle the continuation, but rather some generic handler at the top 
>> level.
> 
> The scheme you have described sounds like an implementation of "escape 
> continuations" -- i.e. continuations which act like exceptions. Such 
> continuations are strictly less powerful than either full continuations 
> or coroutines, IIRC.

First point sounds plausible. Please elaborate the second point 
(perhaps off-list, if you think it is too off-topic).

>> *Storage.* Since [suspend] starts putting data in the interpreter
>> result, everything has to be put under a Tcl_Obj wrapper. This is
>> potentially a slow operation (but probably not so slow that it becomes
>> a major issue when suspending the handler for an event). By contrast,
>> [coroutine]/[yield] hides everything under the opaque handle of a 
>> command.
> 
> The opaque handle nature of coroutines is, I think, acceptable in this 
> case. A coroutine encapsulates rather a complex chunk of interpreter 
> state, so a string representation would be tricky to concoct (but 
> perhaps not impossible).

Ultimately, we may always fall back to what is written on the tape of 
our Turing machine. :-) More practically, also the NRE has to separate 
the per-coroutine state from the general interpreter state, and IMHO 
that is where you might encounter nontrivial issues. The rest should 
just be the hard work of serializing a well-defined data structure.

> That state also changes every time the 
> coroutine is invoked. The fact that coroutines automatically clean-up 
> the command when they are "exhausted" also seems to reduce any 
> difficulties with an opaque naming scheme. Creating uniquely-named 
> coroutines in a special namespace (such as on the wiki page I linked 
> above) will be mostly good enough. The case it misses is where a 
> coroutine is created and then discarded before it has been allowed to 
> run to completion. In this case an explicit clean-up would be needed.

It's certainly possible to live with, but I can't help finding the need 
for this temporary command rather ugly. Then again, I find most APIs 
that create objects to keep track of pure data ugly too.

> While I agree with the sentiment that it would be nice to have Tcl_Obj 
> based representation of coroutines or some form of continuation, I think 
> in practice it would be a lot of work for perhaps not a huge amount of 
> gain. Opaque works for threads and interps; I think it will suffice for 
> coroutines too.

The difference is that threads and interps (at least if you're running 
the event loop) have an internal state that can mutate while you're not 
looking. Coroutines are (I hope) immutable when they're not running.

> Without a more thorough understanding of how suspend/resume are to work, 
> I can't assess whether that really works or not (and thus whether 
> suspend/resume offer more than escape continuations). I would guess it 
> is at least much slower than the coroutine equivalent, having to 
> save/restore command invocations from string reps.

Remember that we have Tcl_Objs, and can stash data in the internal 
representation. The string rep wouldn't normally be generated, it is 
sufficient that it can be.

That said, it is probably not possible to make [suspend] as fast as 
[yield], but it is also more general.

> How does a Tcl proc get to restore itself on a [resume]? I don't really 
> follow how [resume] works at all, to be honest.

There was a detail missing from my sketch: the continuation must record 
what command the resumption is to begin with; sorry about that. A 
proof-of-concept implementation of [suspend] and [resume] can be found 
at http://wiki.tcl.tk/21538.

Lars Hellström