[tclquadcode] More on callframe refactoring.

[Kevin K. <kev...@gm...>]

Donal,

As you know, I'm working on refactoring the moveToCallFrame and
moveFromCallFrame stuff to be amenable to better analysis.

The first step is to get everything running correctly if atrociously
slowly, by making all references to Tcl variables access the copies in
the callframe. (Then I can put the optimizations back in, and have a
better theory of safety, that can even possibly be amenable to some
traces.) I'm still struggling with getting everything working.

I've managed to reduce the number of test failures in the
'kbk-refactor-callframe' branch substantially, but there is still a
handful of things that I'm grappling with.

====
1. magicreturn.
The 'magicreturn' test is crashing code generation. I think that the
issue is that the case:

    set l 0
    set z [return -level $l [expr {$y + $y}]]

has never tested what it purported to test. By the time we implemented
[return -level], the quadcode front end already had constant folding,
so it generated code similar to the first case. The return options
reduced to {literal {-level 0}}.

Now that $l is being retrieved from the callframe (and the retrieval
is intentionally not yet being optimized) we are generating a code
burst that looks something like:

   moveFromCallFrame {var l} {temp callframe} {literal l}
   list {temp 0} {literal -level} {var l}
   ... argument checking for 'add' ...
   add {temp 1} {temp y} {temp y}
   initException {temp exception} {temp 1} {temp 0}

so that the type signature of the initException will be
        NUMERIC -> STRING -> FAIL NUMERIC

This runs through 'compile' and the Big Switch passes it off to
InitException in compile.tcl. It doesn't hit the pretty '-level 0'
special case, and now (for the first time, perhaps?) gets into the
code that looks like

    my StoreResult $tgt [$b $opcode {*}$vals $value $maintype \
        $errorCode $name]

and it's not finding an appropriately typed initException. Is there
supposed to be some sort of promotion of NUMERIC here, or is there
something missing that's supposed to allow it to promote a NUMERIC to
a FAIL NUMERIC?

====
2. lsetest

lstetest still fails mysteriously, and I haven't yet found an
explanation. I'd appreciate a second set of eyeballs. It's almost
certainly something I'm doing in code gen, but I'm just not seeing it!

====
3. direct*

Because of aliases in the callframe, all the direct* operations at
least notionally need to appear to modify the callframe for data flow
analysis to work. If you could give me a worked example of modifying
one of them (directAppend or directIncr, perhaps), or at least a
sketch of how to proceed, to change from

    directAppend listOrErrorOut listIn value...
    jumpMaybe catchBlock, listOrErrorOut

to

    directAppend combinedOut callframeIn listIn value...
    extractCallFrame callframeOut combinedOut
    retrieveResult listOrErrorOut listIn value...
    jumpMaybe catchBlock, listOrErrorOut

that would be a huge help.  Mostly, I need to learn how to construct
and manage reference counting for the combined result. In the case of
directAppend, the result type will be a CALLFRAME FAIL STRING, and it
isn't clear to me what all the correct accessors are.

I think that http://core.tcl.tk/tclquadcode/tktview?name=58e3e71962
may relate to this issue as well.

(There is also a raft of TODO comments around the direct ops in
translate.tcl, having to do with direct operations on
non-fully-qualified names. Was this something you were leaving for me
to chase? Can we at least have tickets for these, if that's the case?)
====
I'd really like to have the tests stable again before I start plugging
away on doing rudimentary escape analysis, which would then allow us
to attempt all four combinations of elimination of operations:
load-load (the same variable pulled from the callframe or a namespace
twice without anything intervening that might have changed it),
load-store (saving a value to the callframe that is known to be
identical to the value that's already there, and for which any
necessary traces have fired.), store-load (retrieving a value from the
callframe that cannot have changed since we put it there), and
store-store (overwriting a value in the callframe that nothing can
have examined with a same or different value).  Ths should be more
stable and flexible than the current approach of updating the
callframe only at escape points (mostly invokes). It should also allow
some amount of code motion; a load operation inside a loop that does
not modify the variable can be hoisted out of the loop, for instance.

Part of the reason I did pre.tcl is that it provides a good partial
framework in which to implement this sort of thing.  The concepts of
'available value' and 'anticipable value' that it uses generalize
readily to slots in the callframe. The fact that it's working well now
gives me some hope that we can get to intelligent load/store
management.

But I'm really bogged down with this last handful of tests, and really
reluctant to move farther forward until they're stable.

Help?