tcl-core — Tcl Core Team discussions about Tcl development

Re: [TCLCORE] CFV: TIP 733, Add accessibility/screen reader support to Tk

[Jan N. <jan...@gm...>]

Op vr 7 nov 2025 om 21:19 schreef Kevin Walzer:
>
> What test cases are failing?

See:
    <https://github.com/tcltk/tk/actions?query=branch%3Acore-tka11y>


On MacOS:
safe.test
==== safe-1.1 Safe Tk loading into an interpreter FAILED
==== Contents of test case:
safe::loadTk [safe::interpCreate a]
safe::interpDelete a
set x {}
return $x
---- Test generated error; Return code was: 1
---- Return code should have been one of: 0 2
---- errorInfo: Can't find a usable tk.tcl in the following directories:
{$p(:12:)}
.......
Tests ended at 2025-11-07 08:00:25 UTC
all.tcl: Total 10050 Passed 8791 Skipped 1223 Failed 36
Sourced 98 Test Files.
Files with failing tests: safe.test safePrimarySelection.test tk.test


With --disable-aqua:
DYLD_FALLBACK_LIBRARY_PATH="`pwd`:/Users/runner/work/tk/tk/tcl/unix:${DYLD_FALLBACK_LIBRARY_PATH}";
export DYLD_FALLBACK_LIBRARY_PATH;
TCL_LIBRARY=/Users/runner/work/tk/tk/tcl/library; export TCL_LIBRARY;
TK_LIBRARY=/Users/runner/work/tk/tk/tk/library; export TK_LIBRARY;
./tktest /Users/runner/work/tk/tk/tk/unix/../tests/all.tcl
/bin/sh: line 1: 19218 Segmentation fault: 11 ./tktest
/Users/runner/work/tk/tk/tk/unix/../tests/all.tcl
make: *** [test-classic] Error 139
killing process 19155...
Error: Failure during Test (classic)
Error: Process completed with exit code 1.


On Linux, with --enable-symbols:
LD_LIBRARY_PATH="`pwd`:/home/runner/work/tk/tk/tcl/unix:${LD_LIBRARY_PATH}";
export LD_LIBRARY_PATH;
TCL_LIBRARY=/home/runner/work/tk/tk/tcl/library; export TCL_LIBRARY;
TK_LIBRARY=/home/runner/work/tk/tk/tk/library; export TK_LIBRARY;
./tktest /home/runner/work/tk/tk/tk/unix/../tests/all.tcl
make: *** [Makefile:734: test-classic] Segmentation fault (core dumped)
Error: Process completed with exit code 2.

Hope this helps,
     Jan Nijtmans

Re: [TCLCORE] CFV: TIP 733, Add accessibility/screen reader support to Tk

[Kevin W. <kw...@co...>]

Found them. They appear to relate to some changes that someone else - a separate bugfix branch has similar failures. Do I need to wait to merge even though I am not responsible for these failures?

> On Nov 7, 2025, at 3:20 PM, Kevin Walzer <kw...@co...> wrote:
> 
> 
> What test cases are failing?
> 
>>> On Nov 7, 2025, at 3:04 PM, Jan Nijtmans <jan...@gm...> wrote:
>>> 
>> 
>> 
>> 
>> Op do 6 nov 2025, 15:35 schreef Jan Nijtmans:
>>> TIP 733: YES
>> 
>> 
>> That said, Ik don't think the branch is ready to be merged, testcases are still failing. But it looks solvabel.
>> 
>> Regards,
>>        Jan Nijtmans
>> _______________________________________________
>> Tcl-Core mailing list
>> Tcl...@li...
>> https://lists.sourceforge.net/lists/listinfo/tcl-core
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] [Ext] Re: [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)

[EricT <tw...@gm...>]

Rene,

That's the beauty of making bytecode compilation official - we DON'T have
to agree on syntax! With a supported tcl::bytecode API, each of us can
create our own Design-Specific Language for our own needs. No more "one
size fits all" debates that go nowhere.

That said, I think you might find Colin's approach already fits your needs
quite well. Here's how your examples would look:

Multiple assignments:

# Your let syntax
let {x 1+2 y 3*4 z $x+$y}

# Colin's syntax (bare variables, no $)
: {x = 1+2
   y = 3*4
   z = x+y}


For my own taste, I've here aliased = to : since = as the command seems to
clash visually with = for assignment, especially in cases like a = b = c =
d. The semicolon requires braces since it's special to Tcl, but by mapping
newlines to semicolons internally, multiple statements work naturally in
braced blocks as shown above.

Returning multiple values:

# With a simple helper function
proc tcl::mathfunc::gather {args} { list {*}$args }

# Your canvas example
.c create text {*}[= gather(x+1, y+1)] -text a

# Or inline
= {tx = x+1 ; ty = y+1 ; gather(tx, ty)}


Your calculations example:

set i 0.5
= {x = sin(i)
   y = cos(i) + x
   z = x + y}


The key differences: bare variables (no $), explicit = for assignment,
semicolon (or newline) to separate multiple expressions, and gather() for
multiple returns. But it's all extensible - add your own functions to
tcl::mathfunc:: and they're automatically available.

If you prefer different syntax, you can build your let command using the
same bytecode infrastructure. That's the whole point - multiple DSLs
coexisting, each optimized by bytecode caching.


Eric

On Fri, Nov 7, 2025 at 6:19 AM Zaumseil René via Tcl-Core <
tcl...@li...> wrote:

> Hi Eric
>
>
>
> I really like the tcl::unsupported::assemble approach.
>
> It opens the way to add more commands/sugar to tcl itself without
>
> having to hard code it in C.
>
>
>
> Now we have to agree on the syntax. May be a wiki page with a summary of
> the different proposals?
>
>
>
>
>
> Regards
>
> rene
>
>
>
>
>
> *Von:* EricT <tw...@gm...>
> *Gesendet:* Freitag, 7. November 2025 01:03
> *An:* Zaumseil René <RZa...@kk...>; tcl...@li...
> *Betreff:* Re: [TCLCORE] [Ext] Re: [=] for concise expressions (was Re:
> TIP 672 Implementation Complete - Ready for Sponsorship)
>
>
>
> Hi Rene,
>
>
>
> I was experimenting with your TIP 674 prototype and realized both your let command and Colin's expression evaluator share something important: they could both be implemented in pure Tcl if tcl::unsupported::assemble were promoted to supported status with handle-based caching.
>
>
>
> This would avoid another round of "expr wars" where lack of consensus leaves us with nothing. Instead of debating whose syntax is better, we'd each have the power to create our own Design-Specific Language. With bytecode caching, pure Tcl implementations achieve C extension performance - Colin's evaluator runs at 1.8 microseconds, nearly as fast as expr.
>
>
>
> Why does this matter for your TIP? If let performs just as well in Tcl as in C, why require a C implementation at all? C extensions take weeks to develop, need binaries for every platform/architecture combination, and ultimately call the same bytecode infrastructure that assemble uses. A supported bytecode API would make both proposals - and many others - viable without C.
>
>
>
> Perhaps we should advocate together for making the bytecode compilation infrastructure officially supported, rather than competing for TIP approval on individual commands?
>
>
>
> Best,
>
> Eric
>
>
>
>
>
> On Wed, Nov 5, 2025 at 11:31 PM Zaumseil René via Tcl-Core <
> tcl...@li...> wrote:
>
> Hello
>
>
>
> I also like the = approach as a new expr command without need of $ for
> variables.
>
> But I'm not sure about the currently proposed syntax.
>
> Is it "= 1 + 2" or "= 1+2" (with or without spaces)?
>
> If it is the later then I would propose to use some of the syntax of the
> "let" from tip 674.
>
>
>
> Some syntax ideas:
>
> "= 1+2" return 3, simple case
>
> "= a 1+2 b 2+4 c a+b" return {3 6 9} and set a,b,c
>
> "= 1+2 = 2+4" return {3 6}
>
> And may be some others…
>
>
>
> Regards
>
> rene
>
> *Von:* EricT <tw...@gm...>
> *Gesendet:* Mittwoch, 5. November 2025 23:22
> *An:* Colin Macleod <col...@ya...>;
> tcl...@li...
> *Betreff:* [Ext] Re: [TCLCORE] [=] for concise expressions (was Re: TIP
> 672 Implementation Complete - Ready for Sponsorship)
>
>
>
> Hi Colin,
>
>
>
> I've successfully modified your amazing code to handle arrays. In doing so, I also found 2 other issues, one is with your Boolean check, the other with your function name check, both because of [string is] issues.
>
>
>
> - Boolean check: `$token eq "false" || $token eq "true"` (was `[string is boolean $token]` - treated 'f','n', 't', 'y', etc. variables as boolean false, no, true, yes, ...)
>
>
>
> - Function check: `[regexp {^[[:alpha:]]} $token]` (was `[string is alpha $token]` - broke log10, atan2)
>
>
>
>
>
> here's the code for arrays:
>
>
>
>     # Function call or array reference?
>
>     set nexttok [lindex $::tokens $::tokpos]
>
>     if {$nexttok eq "(" && [regexp {^[[:alpha:]]} $token]} {
>
>         set fun [namespace which tcl::mathfunc::$token]
>
>         if {$fun ne {}} {
>
>             # It's a function
>
>             incr ::tokpos
>
>             set opcodes "push $fun; "
>
>             append opcodes [parseFuncArgs]
>
>             return $opcodes
>
>         } else {
>
>             # Not a function, assume array reference
>
>             incr ::tokpos
>
>             set opcodes "push $token; "
>
>             # Parse the index expression - leaves VALUE on stack
>
>             append opcodes [parse 0]
>
>             # Expect closing paren
>
>             set closing [lindex $::tokens $::tokpos]
>
>             if {$closing ne ")"} {
>
>                 error "Calc: expected ')' but found '$closing'"
>
>             }
>
>             # Stack now has: [arrayname, indexvalue]
>
>             incr ::tokpos
>
>             append opcodes "loadArrayStk; "
>
>             return $opcodes
>
>         }
>
>     }
>
>
>
>
>
> In addition, there has indeed been some changes in the bytecode, land and lor are no longer supported in 9.0 although they work in 8.6.
>
>
>
> I had an AI generate some 117 test cases, which all pass on 8.6 and 111 on 9.x (the land/lor not being tested in 9.x).
>
>
>
> Colin, with your permission, I can post the code as a new file, with all the test cases, say on a repository at github.
>
>
>
> I think a new TIP is worth considering; one that promotes assemble to a supported form, with a compile and handle approach to avoid the time parsing the ascii byte code text. I think that this would be great for your = command, but also quite useful for others who might want to create their own little languages.
>
>
>
> By doing it this way, it remains pure tcl, and avoids all the problems with different systems and hardware that a binary extension would create. In the end, I believe your code can achieve performance parity with expr. Not only does it remove half the [expr {...}] baggage, but all the $'s too! So much easier on these old eyes.
>
>
>
> Regards,
>
>
>
> Eric
>
>
>
>
>
> On Tue, Nov 4, 2025 at 1:06 PM EricT <tw...@gm...> wrote:
>
> Hi Colin,
>
>
>
> Hmmm, why can't you do bareword on $a(b) as a(b) you just need to do an
> uplevel to see if a is a variable, if not, it would have to be a function.
> True?
>
>
>
> % tcl::unsupported::disassemble script {set a [expr {$b($c)}] }
> snip
>
>   Command 2: "expr {$b($c)}..."
>     (2) push1 1 # "b"
>     (4) push1 2 # "c"
>     (6) loadStk
>     (7) loadArrayStk
>     (8) tryCvtToNumeric
>     (9) storeStk
>     (10) done
>
> This doesn't look too much different from what you are producing.
>
>
>
> I think what's really needed here is a TIP that would open up the bytecode
> a bit so you don't need to use an unsupported command. And then maybe even
> have a new command to take the string byte code you are now producing and
> return a handle to a cached version that was probably equivalent to the
> existing bytecode. Then your cache array would be
>
>
>
>  set cache($exp) $handle
>
>
>
> Instead of it having to parse the text, it could be as fast as bytecode.
> You'd likely be just as fast as expr, and safe as well, since you can't
> pass a string command in where the bareword is required:
>
>
>
>  % set x {[pwd]}
> [pwd]
>  % = sqrt(x)
> exp= |sqrt(x)| code= |push ::tcl::mathfunc::sqrt; push x; loadStk;
> invokeStk 2; | ifexist: 0
> expected floating-point number but got "[pwd]"
>
> I think you really have something here, perhaps this is the best answer
> yet to slay the expr dragon!
>
>
>
> Regards,
>
>
>
> Eric
>
>
>
>
>
> On Tue, Nov 4, 2025 at 6:52 AM Colin Macleod via Tcl-Core <
> tcl...@li...> wrote:
>
> Hi Eric,
>
> That's very neat!
>
> Yes, a pure Tcl version could go into TclLib. I still think it may be
> worth trying a C implementation though.  The work-around that's needed for
> array references [= 2* $a(b)] would defeat the caching, so it would be good
> to speed up the parsing if possible.  Also I think your caching may be
> equivalent to doing byte-compilation, in which case it may make sense to
> use the framework which already exists for that.
>
> Colin.
>
> On 04/11/2025 01:18, EricT wrote:
>
> that is:
>
>
>
> if {[info exist ::cache($exp)]} {
>
>     tailcall ::tcl::unsupported::assemble $::cache($exp)
> }
>
>
>
> (hate gmail!)
>
>
>
>
>
> On Mon, Nov 3, 2025 at 5:17 PM EricT <tw...@gm...> wrote:
>
> and silly of me, it should be:
> if {[info exist ::cache($exp)]} {
>     tailcall ::tcl::unsupported::assemble $::cache($exp)
> }
>
>
>
> On Mon, Nov 3, 2025 at 4:50 PM EricT <tw...@gm...> wrote:
>
> With a debug line back in plus the tailcall:
>
>
>
> proc = args {
>
>     set exp [join $args]
>     if { [info exist ::cache($exp)] } {
>          return [tailcall  ::tcl::unsupported::assemble $::cache($exp)]
>     }
>     set tokens [tokenise $exp]
>     deb1 "TOKENS = '$tokens'"
>     set code [compile $tokens]
>     deb1 "GENERATED CODE:\n$code\n"
>     puts "exp= |$exp| code= |$code| ifexist: [info exist ::cache($exp)]"
>     set ::cache($exp) $code
>     uplevel [list ::tcl::unsupported::assemble $code]
> }
>
>
>
> % set a 5
> 5
>  % set b 10
> 10
>  % = a + b
> exp= |a + b| code= |push a; loadStk; push b; loadStk; add; | ifexist: 0
> 15
> % = a + b
> 15
>
> % time {= a + b} 1000
> 1.73 microseconds per iteration
>
> Faster still!
>
>
>
> I thought the uplevel was needed to be able to get the local variables,
> seems not.
>
>  % proc foo arg {set a 5; set b 10; set c [= a+b+arg]}
>  % foo 5
> exp= |a+b+arg| code= |push a; loadStk; push b; loadStk; add; push arg;
> loadStk; add; | ifexist: 0
> 20
> % foo 5
> 20
>
>
>
>  % proc foo arg {global xxx; set a 5; set b 10; set c [= a+b+arg+xxx]}
>
>  % set xxx 100
> 100
>  % foo 200
> 315
>  % time {foo 200} 10000
> 2.1775 microseconds per iteration
>
>
>
>  % parray cache
>
> cache(a + b)       = push a; loadStk; push b; loadStk; add;
> cache(a+b+arg)     = push a; loadStk; push b; loadStk; add; push arg;
> loadStk; add;
> cache(a+b+arg+xxx) = push a; loadStk; push b; loadStk; add; push arg;
> loadStk; add; push xxx; loadStk; add;
>
>
>
> Very Impressive, great job Colin! Great catch Don!
>
>
>
> Eric
>
>
>
>
>
>
>
>
>
> On Mon, Nov 3, 2025 at 4:22 PM Donald Porter via Tcl-Core <
> tcl...@li...> wrote:
>
> Check what effect replacing [uplevel] with [tailcall] has.
>
>
>
> On Nov 3, 2025, at 7:13 PM, EricT <tw...@gm...> wrote:
>
>
>
> Subject: Your bytecode expression evaluator - impressive results with
> caching!
>
> Hey Colin:
>
> I took a look at your bytecode-based expression evaluator and was
> intrigued by the approach. I made a small modification to add caching and
> the results are really impressive. Here's what I changed:
>
> proc = args {
>     set exp [join $args]
>     if {[info exist ::cache($exp)]} {
>         return [uplevel [list ::tcl::unsupported::assemble $::cache($exp)]]
>     }
>     set tokens [tokenise $exp]
>     deb1 "TOKENS = '$tokens'"
>     set code [compile $tokens]
>     deb1 "GENERATED CODE:\n$code\n"
>     set ::cache($exp) $code
>     uplevel [list ::tcl::unsupported::assemble $code]
> }
>
> The cache is just a simple array lookup - one line to store, one line to
> retrieve. But the performance impact is huge:
>
> Performance Tests
>
> Without caching
> % time {= 1 + 2} 1000
> 24.937 microseconds per iteration
>
> With caching
> % time {= 1 + 2} 1000
> 1.8 microseconds per iteration
>
> That's a 13x speedup! The tokenize and parse steps were eating about 92%
> of the execution time.
>
> The Real Magic: Bare Variables + Caching
>
> What really impressed me is how well your bare variable feature synergizes
> with caching:
>
> % set a 5
> 5
> % set b 6
> 6
> % = a + b
> 11
> % time {= a + b} 1000
> 2.079 microseconds per iteration
>
> Now change the variable values
> % set a 10
> 10
> % = a + b
> 16
> % time {= a + b} 1000
> 2.188 microseconds per iteration
>
> The cache entry stays valid even when the variable values change! Why?
> Because the bytecode stores variable names, not values:
>
> push a; loadStk; push b; loadStk; add;
>
> The loadStk instruction does runtime lookup, so:
> - Cache key is stable: "a + b"
> - Works for any values of a and b
> - One cache entry handles all value combinations
>
> Compare this to if we used $-substitution:
>
> = $a + $b    # With a=5, b=6 becomes "5 + 6"
> = $a + $b    # With a=10, b=6 becomes "10 + 6" - different cache key!
>
> Every value change would create a new cache entry or worse, a cache miss.
>
> Comparison to Other Approaches
>
> Tcl's expr:              about 0.40 microseconds
> Direct C evaluator:      about 0.53 microseconds
> Your cached approach:    about 1.80 microseconds
> Your uncached approach:  about 24.9 microseconds
>
> With caching, you're only 3-4x slower than a direct C evaluator.
>
>
> My Assessment
>
> Your design is excellent. The bare variable feature isn't just syntax
> sugar - it's essential for good cache performance. The synergy between:
>
> 1. Bare variables leading to stable cache keys
> 2. Runtime lookup keeping cache hot
> 3. Simple caching providing dramatic speedup
>
> makes this really elegant.
>
> My recommendation: Keep it in Tcl! The implementation is clean,
> performance is excellent (1.8 microseconds is plenty fast), and converting
> to C would add significant complexity for minimal gain (maybe getting to
> about 1.0 microseconds).
>
> The Tcl prototype with caching is actually the right solution here.
> Sometimes the prototype IS the product!
>
> Excellent work on this. The bytecode approach really shines with caching
> enabled.
>
>
>
> On Sun, Nov 2, 2025 at 10:14 AM Colin Macleod via Tcl-Core <
> tcl...@li...> wrote:
>
> Hi again,
>
> I've now made a slightly more serious prototype, see
> https://cmacleod.me.uk/tcl/expr_ng
>
> This is a modified version of the prototype I wrote for tip 676. It's
> still in Tcl, but doesn't use `expr`.  It tokenises and parses the input,
> then generates TAL bytecode and uses ::tcl::unsupported::assemble to run
> that.  A few examples:
>
> (bin) 100 % set a [= 3.0/4]
> 0.75
> (bin) 101 % set b [= sin(a*10)]
> 0.9379999767747389
> (bin) 102 % set c [= (b-a)*100]
> 18.79999767747389
> (bin) 103 % namespace eval nn {set d [= 10**3]}
> 1000
> (bin) 104 % set e [= a?nn::d:b]
> 1000
> (bin) 105 % = {3 + [pwd]}
> Calc: expected start of expression but found '[pwd]'
> (bin) 106 % = {3 + $q}
> Calc: expected start of expression but found '$q'
> (bin) 107 % = sin (12)
> -0.5365729180004349
>
> (bin) 108 % array set rr {one 1 two 2 three 3}
> (bin) 110 % = a * rr(two)
> Calc: expected operator but found '('
> (bin) 111 % = a * $rr(two)
> 1.5
>
> - You can use $ to get an array value substituted before the `=` code sees
> the expression.
>
> (bin) 112 % string repeat ! [= nn::d / 15]
> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>
> Colin.
>
> On 02/11/2025 09:04, Donal Fellows wrote:
>
> Doing the job properly would definitely involve changing the expression
> parser, with my suggested fix being to turn all bare words not otherwise
> recognised as constants or in positions that look like function calls (it's
> a parser with some lookahead) into simple variable reads (NB: C resolves
> such ambiguities within itself differently, but that's one of the nastiest
> parts of the language). We would need to retain $ support for resolving
> ambiguity (e.g., array reads vs function calls; you can't safely inspect
> the interpreter to resolve it at the time of compiling the expression due
> to traces and unknown handlers) as well as compatibility, but that's doable
> as it is a change only in cases that are currently errors.
>
>
>
> Adding assignment is quite a bit trickier, as that needs a new major
> syntax class to describe the left side of the assignment. I suggest
> omitting that from consideration at this stage.
>
>
>
> Donal.
>
>
>
> -------- Original message --------
>
> From: Colin Macleod via Tcl-Core <tcl...@li...>
> <tcl...@li...>
>
> Date: 02/11/2025 08:13 (GMT+00:00)
>
> To: Pietro Cerutti <ga...@ga...> <ga...@ga...>
>
> Cc: tcl...@li..., av...@lo...
>
> Subject: Re: [TCLCORE] Fwd: TIP 672 Implementation Complete - Ready for
> Sponsorship
>
>
>
> Indeed, this toy implementation doesn't handle that:
>
> % = sin (12)
> can't read "sin": no such variable
>
> I'm not sure that's serious, but it could be fixed in a C implementation.
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>
>
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>
>
>
> _______________________________________________
>
> Tcl-Core mailing list
>
> Tcl...@li...
>
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>

Re: [TCLCORE] CFV: TIP 733, Add accessibility/screen reader support to Tk

[Kevin W. <kw...@co...>]

What tests are failing?

> On Nov 7, 2025, at 3:04 PM, Jan Nijtmans <jan...@gm...> wrote:
> 
> 
> 
> 
> Op do 6 nov 2025, 15:35 schreef Jan Nijtmans:
>> TIP 733: YES
> 
> 
> That said, Ik don't think the branch is ready to be merged, testcases are still failing. But it looks solvabel.
> 
> Regards,
>        Jan Nijtmans
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] CFV: TIP 733, Add accessibility/screen reader support to Tk

[Kevin W. <kw...@co...>]

What test cases are failing?

> On Nov 7, 2025, at 3:04 PM, Jan Nijtmans <jan...@gm...> wrote:
> 
> 
> 
> 
> Op do 6 nov 2025, 15:35 schreef Jan Nijtmans:
>> TIP 733: YES
> 
> 
> That said, Ik don't think the branch is ready to be merged, testcases are still failing. But it looks solvabel.
> 
> Regards,
>        Jan Nijtmans
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] CFV: TIP 733, Add accessibility/screen reader support to Tk

[Jan N. <jan...@gm...>]

Op do 6 nov 2025, 15:35 schreef Jan Nijtmans:

> TIP 733: YES
>

That said, Ik don't think the branch is ready to be merged, testcases are
still failing. But it looks solvabel.

Regards,
       Jan Nijtmans

>

Re: [TCLCORE] CFV: TIP 733, Add accessibility/screen reader support to Tk

[Andreas K. <and...@gm...>]

> Hi all,
> 
> I've had no additional feedback on TIP 733 in the past week, and I 
> believe the tka11y branch is ready for merging, so I'd like to call for 
> a TCT vote.
> 
> Let's have votes in by [clock format 1762559940 -gmt 1].
> 
> My vote: TIP 733: YES

TIP#733	YES

-- 
Happy Tcling,
	Andreas Kupries <and...@gm...>
			<https://core.tcl-lang.org/akupries/>
			<https://akupries.tclers.tk/>
	Developer @	SUSE Software Solutions Germany GmbH
-------------------------------------------------------------------------------

Re: [TCLCORE] [Ext] Re: [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)

[Zaumseil R. <RZa...@kk...>]

Hi Eric

I really like the tcl::unsupported::assemble approach.
It opens the way to add more commands/sugar to tcl itself without
having to hard code it in C.

Now we have to agree on the syntax. May be a wiki page with a summary of the different proposals?


Regards
rene


Von: EricT <tw...@gm...>
Gesendet: Freitag, 7. November 2025 01:03
An: Zaumseil René <RZa...@kk...>; tcl...@li...
Betreff: Re: [TCLCORE] [Ext] Re: [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)


Hi Rene,



I was experimenting with your TIP 674 prototype and realized both your let command and Colin's expression evaluator share something important: they could both be implemented in pure Tcl if tcl::unsupported::assemble were promoted to supported status with handle-based caching.



This would avoid another round of "expr wars" where lack of consensus leaves us with nothing. Instead of debating whose syntax is better, we'd each have the power to create our own Design-Specific Language. With bytecode caching, pure Tcl implementations achieve C extension performance - Colin's evaluator runs at 1.8 microseconds, nearly as fast as expr.



Why does this matter for your TIP? If let performs just as well in Tcl as in C, why require a C implementation at all? C extensions take weeks to develop, need binaries for every platform/architecture combination, and ultimately call the same bytecode infrastructure that assemble uses. A supported bytecode API would make both proposals - and many others - viable without C.



Perhaps we should advocate together for making the bytecode compilation infrastructure officially supported, rather than competing for TIP approval on individual commands?



Best,

Eric


On Wed, Nov 5, 2025 at 11:31 PM Zaumseil René via Tcl-Core <tcl...@li...<mailto:tcl...@li...>> wrote:
Hello

I also like the = approach as a new expr command without need of $ for variables.
But I'm not sure about the currently proposed syntax.
Is it "= 1 + 2" or "= 1+2" (with or without spaces)?
If it is the later then I would propose to use some of the syntax of the "let" from tip 674.

Some syntax ideas:
"= 1+2" return 3, simple case
"= a 1+2 b 2+4 c a+b" return {3 6 9} and set a,b,c
"= 1+2 = 2+4" return {3 6}
And may be some others…

Regards
rene
Von: EricT <tw...@gm...<mailto:tw...@gm...>>
Gesendet: Mittwoch, 5. November 2025 23:22
An: Colin Macleod <col...@ya...<mailto:col...@ya...>>; tcl...@li...<mailto:tcl...@li...>
Betreff: [Ext] Re: [TCLCORE] [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)


Hi Colin,



I've successfully modified your amazing code to handle arrays. In doing so, I also found 2 other issues, one is with your Boolean check, the other with your function name check, both because of [string is] issues.



- Boolean check: `$token eq "false" || $token eq "true"` (was `[string is boolean $token]` - treated 'f','n', 't', 'y', etc. variables as boolean false, no, true, yes, ...)



- Function check: `[regexp {^[[:alpha:]]} $token]` (was `[string is alpha $token]` - broke log10, atan2)





here's the code for arrays:



    # Function call or array reference?

    set nexttok [lindex $::tokens $::tokpos]

    if {$nexttok eq "(" && [regexp {^[[:alpha:]]} $token]} {

        set fun [namespace which tcl::mathfunc::$token]

        if {$fun ne {}} {

            # It's a function

            incr ::tokpos

            set opcodes "push $fun; "

            append opcodes [parseFuncArgs]

            return $opcodes

        } else {

            # Not a function, assume array reference

            incr ::tokpos

            set opcodes "push $token; "

            # Parse the index expression - leaves VALUE on stack

            append opcodes [parse 0]

            # Expect closing paren

            set closing [lindex $::tokens $::tokpos]

            if {$closing ne ")"} {

                error "Calc: expected ')' but found '$closing'"

            }

            # Stack now has: [arrayname, indexvalue]

            incr ::tokpos

            append opcodes "loadArrayStk; "

            return $opcodes

        }

    }





In addition, there has indeed been some changes in the bytecode, land and lor are no longer supported in 9.0 although they work in 8.6.



I had an AI generate some 117 test cases, which all pass on 8.6 and 111 on 9.x (the land/lor not being tested in 9.x).



Colin, with your permission, I can post the code as a new file, with all the test cases, say on a repository at github.



I think a new TIP is worth considering; one that promotes assemble to a supported form, with a compile and handle approach to avoid the time parsing the ascii byte code text. I think that this would be great for your = command, but also quite useful for others who might want to create their own little languages.



By doing it this way, it remains pure tcl, and avoids all the problems with different systems and hardware that a binary extension would create. In the end, I believe your code can achieve performance parity with expr. Not only does it remove half the [expr {...}] baggage, but all the $'s too! So much easier on these old eyes.



Regards,



Eric


On Tue, Nov 4, 2025 at 1:06 PM EricT <tw...@gm...<mailto:tw...@gm...>> wrote:
Hi Colin,

Hmmm, why can't you do bareword on $a(b) as a(b) you just need to do an uplevel to see if a is a variable, if not, it would have to be a function. True?

% tcl::unsupported::disassemble script {set a [expr {$b($c)}] }
snip
  Command 2: "expr {$b($c)}..."
    (2) push1 1 # "b"
    (4) push1 2 # "c"
    (6) loadStk
    (7) loadArrayStk
    (8) tryCvtToNumeric
    (9) storeStk
    (10) done
This doesn't look too much different from what you are producing.

I think what's really needed here is a TIP that would open up the bytecode a bit so you don't need to use an unsupported command. And then maybe even have a new command to take the string byte code you are now producing and return a handle to a cached version that was probably equivalent to the existing bytecode. Then your cache array would be

 set cache($exp) $handle

Instead of it having to parse the text, it could be as fast as bytecode. You'd likely be just as fast as expr, and safe as well, since you can't pass a string command in where the bareword is required:

 % set x {[pwd]}
[pwd]
 % = sqrt(x)
exp= |sqrt(x)| code= |push ::tcl::mathfunc::sqrt; push x; loadStk; invokeStk 2; | ifexist: 0
expected floating-point number but got "[pwd]"
I think you really have something here, perhaps this is the best answer yet to slay the expr dragon!

Regards,

Eric


On Tue, Nov 4, 2025 at 6:52 AM Colin Macleod via Tcl-Core <tcl...@li...<mailto:tcl...@li...>> wrote:

Hi Eric,

That's very neat!

Yes, a pure Tcl version could go into TclLib. I still think it may be worth trying a C implementation though.  The work-around that's needed for array references [= 2* $a(b)] would defeat the caching, so it would be good to speed up the parsing if possible.  Also I think your caching may be equivalent to doing byte-compilation, in which case it may make sense to use the framework which already exists for that.

Colin.
On 04/11/2025 01:18, EricT wrote:
that is:

if {[info exist ::cache($exp)]} {
    tailcall ::tcl::unsupported::assemble $::cache($exp)
}

(hate gmail!)


On Mon, Nov 3, 2025 at 5:17 PM EricT <tw...@gm...<mailto:tw...@gm...>> wrote:
and silly of me, it should be:
if {[info exist ::cache($exp)]} {
    tailcall ::tcl::unsupported::assemble $::cache($exp)
}

On Mon, Nov 3, 2025 at 4:50 PM EricT <tw...@gm...<mailto:tw...@gm...>> wrote:
With a debug line back in plus the tailcall:

proc = args {
    set exp [join $args]
    if { [info exist ::cache($exp)] } {
         return [tailcall  ::tcl::unsupported::assemble $::cache($exp)]
    }
    set tokens [tokenise $exp]
    deb1 "TOKENS = '$tokens'"
    set code [compile $tokens]
    deb1 "GENERATED CODE:\n$code\n"
    puts "exp= |$exp| code= |$code| ifexist: [info exist ::cache($exp)]"
    set ::cache($exp) $code
    uplevel [list ::tcl::unsupported::assemble $code]
}

% set a 5
5
 % set b 10
10
 % = a + b
exp= |a + b| code= |push a; loadStk; push b; loadStk; add; | ifexist: 0
15
% = a + b
15

% time {= a + b} 1000
1.73 microseconds per iteration
Faster still!

I thought the uplevel was needed to be able to get the local variables, seems not.

 % proc foo arg {set a 5; set b 10; set c [= a+b+arg]}
 % foo 5
exp= |a+b+arg| code= |push a; loadStk; push b; loadStk; add; push arg; loadStk; add; | ifexist: 0
20
% foo 5
20

 % proc foo arg {global xxx; set a 5; set b 10; set c [= a+b+arg+xxx]}

 % set xxx 100
100
 % foo 200
315
 % time {foo 200} 10000
2.1775 microseconds per iteration

 % parray cache
cache(a + b)       = push a; loadStk; push b; loadStk; add;
cache(a+b+arg)     = push a; loadStk; push b; loadStk; add; push arg; loadStk; add;
cache(a+b+arg+xxx) = push a; loadStk; push b; loadStk; add; push arg; loadStk; add; push xxx; loadStk; add;

Very Impressive, great job Colin! Great catch Don!

Eric




On Mon, Nov 3, 2025 at 4:22 PM Donald Porter via Tcl-Core <tcl...@li...<mailto:tcl...@li...>> wrote:
Check what effect replacing [uplevel] with [tailcall] has.

On Nov 3, 2025, at 7:13 PM, EricT <tw...@gm...<mailto:tw...@gm...>> wrote:

Subject: Your bytecode expression evaluator - impressive results with caching!

Hey Colin:

I took a look at your bytecode-based expression evaluator and was intrigued by the approach. I made a small modification to add caching and the results are really impressive. Here's what I changed:

proc = args {
    set exp [join $args]
    if {[info exist ::cache($exp)]} {
        return [uplevel [list ::tcl::unsupported::assemble $::cache($exp)]]
    }
    set tokens [tokenise $exp]
    deb1 "TOKENS = '$tokens'"
    set code [compile $tokens]
    deb1 "GENERATED CODE:\n$code\n"
    set ::cache($exp) $code
    uplevel [list ::tcl::unsupported::assemble $code]
}

The cache is just a simple array lookup - one line to store, one line to retrieve. But the performance impact is huge:

Performance Tests

Without caching
% time {= 1 + 2} 1000
24.937 microseconds per iteration

With caching
% time {= 1 + 2} 1000
1.8 microseconds per iteration

That's a 13x speedup! The tokenize and parse steps were eating about 92% of the execution time.

The Real Magic: Bare Variables + Caching

What really impressed me is how well your bare variable feature synergizes with caching:

% set a 5
5
% set b 6
6
% = a + b
11
% time {= a + b} 1000
2.079 microseconds per iteration

Now change the variable values
% set a 10
10
% = a + b
16
% time {= a + b} 1000
2.188 microseconds per iteration

The cache entry stays valid even when the variable values change! Why? Because the bytecode stores variable names, not values:

push a; loadStk; push b; loadStk; add;

The loadStk instruction does runtime lookup, so:
- Cache key is stable: "a + b"
- Works for any values of a and b
- One cache entry handles all value combinations

Compare this to if we used $-substitution:

= $a + $b    # With a=5, b=6 becomes "5 + 6"
= $a + $b    # With a=10, b=6 becomes "10 + 6" - different cache key!

Every value change would create a new cache entry or worse, a cache miss.

Comparison to Other Approaches

Tcl's expr:              about 0.40 microseconds
Direct C evaluator:      about 0.53 microseconds
Your cached approach:    about 1.80 microseconds
Your uncached approach:  about 24.9 microseconds

With caching, you're only 3-4x slower than a direct C evaluator.


My Assessment

Your design is excellent. The bare variable feature isn't just syntax sugar - it's essential for good cache performance. The synergy between:

1. Bare variables leading to stable cache keys
2. Runtime lookup keeping cache hot
3. Simple caching providing dramatic speedup

makes this really elegant.

My recommendation: Keep it in Tcl! The implementation is clean, performance is excellent (1.8 microseconds is plenty fast), and converting to C would add significant complexity for minimal gain (maybe getting to about 1.0 microseconds).

The Tcl prototype with caching is actually the right solution here. Sometimes the prototype IS the product!

Excellent work on this. The bytecode approach really shines with caching enabled.

On Sun, Nov 2, 2025 at 10:14 AM Colin Macleod via Tcl-Core <tcl...@li...<mailto:tcl...@li...>> wrote:

Hi again,

I've now made a slightly more serious prototype, see https://cmacleod.me.uk/tcl/expr_ng

This is a modified version of the prototype I wrote for tip 676. It's still in Tcl, but doesn't use `expr`.  It tokenises and parses the input, then generates TAL bytecode and uses ::tcl::unsupported::assemble to run that.  A few examples:

(bin) 100 % set a [= 3.0/4]
0.75
(bin) 101 % set b [= sin(a*10)]
0.9379999767747389
(bin) 102 % set c [= (b-a)*100]
18.79999767747389
(bin) 103 % namespace eval nn {set d [= 10**3]}
1000
(bin) 104 % set e [= a?nn::d:b]
1000
(bin) 105 % = {3 + [pwd]}
Calc: expected start of expression but found '[pwd]'
(bin) 106 % = {3 + $q}
Calc: expected start of expression but found '$q'
(bin) 107 % = sin (12)
-0.5365729180004349

(bin) 108 % array set rr {one 1 two 2 three 3}
(bin) 110 % = a * rr(two)
Calc: expected operator but found '('
(bin) 111 % = a * $rr(two)
1.5

- You can use $ to get an array value substituted before the `=` code sees the expression.

(bin) 112 % string repeat ! [= nn::d / 15]
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Colin.
On 02/11/2025 09:04, Donal Fellows wrote:
Doing the job properly would definitely involve changing the expression parser, with my suggested fix being to turn all bare words not otherwise recognised as constants or in positions that look like function calls (it's a parser with some lookahead) into simple variable reads (NB: C resolves such ambiguities within itself differently, but that's one of the nastiest parts of the language). We would need to retain $ support for resolving ambiguity (e.g., array reads vs function calls; you can't safely inspect the interpreter to resolve it at the time of compiling the expression due to traces and unknown handlers) as well as compatibility, but that's doable as it is a change only in cases that are currently errors.

Adding assignment is quite a bit trickier, as that needs a new major syntax class to describe the left side of the assignment. I suggest omitting that from consideration at this stage.

Donal.

-------- Original message --------
From: Colin Macleod via Tcl-Core <tcl...@li...><mailto:tcl...@li...>
Date: 02/11/2025 08:13 (GMT+00:00)
To: Pietro Cerutti <ga...@ga...><mailto:ga...@ga...>
Cc: tcl...@li...<mailto:tcl...@li...>, av...@lo...<mailto:av...@lo...>
Subject: Re: [TCLCORE] Fwd: TIP 672 Implementation Complete - Ready for Sponsorship


Indeed, this toy implementation doesn't handle that:

% = sin (12)
can't read "sin": no such variable

I'm not sure that's serious, but it could be fixed in a C implementation.
_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core
_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core

_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core


_______________________________________________

Tcl-Core mailing list

Tcl...@li...<mailto:Tcl...@li...>

https://lists.sourceforge.net/lists/listinfo/tcl-core
_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core
_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] [Ext] Re: [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)

[Pietro C. <ga...@ga...>]

<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto">On the other hand, the choice of implementation, be it C or Tcl, is quite orthogonal to the feature being in core, and thus from it requiring a TIP.<div><div><br id="lineBreakAtBeginningOfSignature"><div dir="ltr"><div>--&nbsp;</div>Pietro Cerutti<div><span style="background-color: rgba(255, 255, 255, 0); font-size: 17pt;">I've pledged to give&nbsp;</span><span style="background-color: rgba(255, 255, 255, 0); font-size: 17pt;">10% of income to effective charities</span><span style="background-color: rgba(255, 255, 255, 0); font-size: 17pt;">&nbsp;</span><span style="background-color: rgba(255, 255, 255, 0); font-size: 17pt;">and invite you to join me.</span></div><div>https://givingwhatwecan.org</div><div><br></div><div>Sent from a small device - please excuse brevity and typos.<br><div><br></div></div></div><div dir="ltr"><br><blockquote type="cite">On 7 Nov 2025, at 01:03, EricT &lt;tw...@gm...&gt; wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr"><div dir="ltr"><div><pre style="color:rgb(0,0,0);background-color:rgb(255,255,255);font-family:&quot;Consolas&quot;;font-size:14pt;font-weight:normal;font-style:normal;text-decoration-line:none;white-space:pre-wrap">Hi Rene,

I was experimenting with your TIP 674 prototype and realized both your let command and Colin's expression evaluator share something important: they could both be implemented in pure <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">Tcl</span> if <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">tcl</span>::unsupported::assemble were promoted to supported status with handle-based caching.

This would avoid another round of "<span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">expr</span> wars" where lack of consensus leaves us with nothing. Instead of debating whose syntax is better, we'd each have the power to create our own Design-Specific Language. With <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">bytecode</span> caching, pure <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">Tcl</span> implementations achieve C extension performance - Colin's evaluator runs at 1.8 microseconds, nearly as fast as <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">expr</span>.

Why does this matter for your TIP? If let performs just as well in <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">Tcl</span> as in C, why require a C implementation at all? C extensions take weeks to develop, need binaries for every platform/architecture combination, and ultimately call the same <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">bytecode</span> infrastructure that assemble uses. A supported <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">bytecode</span> <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">API</span> would make both proposals - and many others - viable without C.

Perhaps we should advocate together for making the <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">bytecode</span> compilation infrastructure officially supported, rather than competing for TIP approval on individual commands?

Best,
Eric</pre>
<br></div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Wed, Nov 5, 2025 at 11:31 PM Zaumseil René via Tcl-Core &lt;<a href="mailto:tcl...@li...">tcl...@li...</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg-4722256188258047040">





<div lang="DE-CH" style="overflow-wrap: break-word;">
<div class="m_-4722256188258047040WordSection1">
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif">Hello<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif"><u></u>&nbsp;<u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif">I also like the = approach as a new expr command without need of $ for variables.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif">But I'm not sure about the currently proposed syntax.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif">Is it "= 1 + 2" or "= 1+2" (with or without spaces)?<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif">If it is the later then I would propose to use some of the syntax of the "let" from tip 674.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif"><u></u>&nbsp;<u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif">Some syntax ideas:<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif">"= 1+2" return 3, simple case<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif">"= a 1+2 b 2+4 c a+b" return {3 6 9} and set a,b,c<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif">"= 1+2 = 2+4" return {3 6}<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif">And may be some others…<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif"><u></u>&nbsp;<u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif">Regards<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:&quot;Arial&quot;,sans-serif">rene<u></u><u></u></span></p>
<div style="border-width:1pt medium medium;border-style:solid none none;border-color:rgb(225,225,225) currentcolor currentcolor;padding:3pt 0cm 0cm">
<p class="MsoNormal"><b><span lang="DE" style="font-size:11pt;font-family:&quot;Calibri&quot;,sans-serif">Von:</span></b><span lang="DE" style="font-size:11pt;font-family:&quot;Calibri&quot;,sans-serif"> EricT &lt;<a href="mailto:tw...@gm..." target="_blank">tw...@gm...</a>&gt;
<br>
<b>Gesendet:</b> Mittwoch, 5. November 2025 23:22<br>
<b>An:</b> Colin Macleod &lt;<a href="mailto:col...@ya..." target="_blank">col...@ya...</a>&gt;; <a href="mailto:tcl...@li..." target="_blank">tcl...@li...</a><br>
<b>Betreff:</b> [Ext] Re: [TCLCORE] [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)<u></u><u></u></span></p>
</div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
<div>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">Hi Colin,<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u>&nbsp;<u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">I've successfully modified your amazing code to handle arrays. In doing so, I also found 2 other issues, one is with your Boolean check, the other with your function name check, both because of [string is] issues. <u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u>&nbsp;<u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">- Boolean check: `$token <span class="m_-4722256188258047040gmail-syntax9">eq</span> "false" || $token <span class="m_-4722256188258047040gmail-syntax9">eq</span> "true"` (was `[string is <span class="m_-4722256188258047040gmail-syntax9">boolean</span> $token]` - treated 'f','n', 't', 'y', <span class="m_-4722256188258047040gmail-syntax9">etc</span>. variables as <span class="m_-4722256188258047040gmail-syntax9">boolean</span> false, no, true, yes, ...)<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u>&nbsp;<u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">- Function check: `[regexp {^[[:alpha:]]} $token]` (was `[string is alpha $token]` - broke log10, atan2)<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u>&nbsp;<u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u>&nbsp;<u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">here's the code for arrays:<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u>&nbsp;<u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp; # Function call or array reference?<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp; set <span class="m_-4722256188258047040gmail-syntax9">nexttok</span> [<span class="m_-4722256188258047040gmail-syntax9">lindex</span> $::tokens $::<span class="m_-4722256188258047040gmail-syntax9">tokpos</span>]<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp; if {$<span class="m_-4722256188258047040gmail-syntax9">nexttok</span> <span class="m_-4722256188258047040gmail-syntax9">eq</span> "(" &amp;&amp; [regexp {^[[:alpha:]]} $token]} {<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; set fun [<span class="m_-4722256188258047040gmail-syntax9">namespace</span> which <span class="m_-4722256188258047040gmail-syntax9">tcl</span>::<span class="m_-4722256188258047040gmail-syntax9">mathfunc</span>::$token]<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if {$fun ne {}} {<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; # It's a function<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <span class="m_-4722256188258047040gmail-syntax9">incr</span> ::<span class="m_-4722256188258047040gmail-syntax9">tokpos</span><u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; set <span class="m_-4722256188258047040gmail-syntax9">opcodes</span> "push $fun; "<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; append <span class="m_-4722256188258047040gmail-syntax9">opcodes</span> [<span class="m_-4722256188258047040gmail-syntax9">parseFuncArgs</span>]<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return $<span class="m_-4722256188258047040gmail-syntax9">opcodes</span><u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; } else {<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; # Not a function, assume array reference<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <span class="m_-4722256188258047040gmail-syntax9">incr</span> ::<span class="m_-4722256188258047040gmail-syntax9">tokpos</span><u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; set <span class="m_-4722256188258047040gmail-syntax9">opcodes</span> "push $token; "<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; # Parse the index expression - leaves VALUE on stack<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; append <span class="m_-4722256188258047040gmail-syntax9">opcodes</span> [parse 0]<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; # Expect closing <span class="m_-4722256188258047040gmail-syntax9">paren</span><u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; set closing [<span class="m_-4722256188258047040gmail-syntax9">lindex</span> $::tokens $::<span class="m_-4722256188258047040gmail-syntax9">tokpos</span>]<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if {$closing ne ")"} {<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; error "<span class="m_-4722256188258047040gmail-syntax9">Calc</span>: expected ')' but found '$closing'"<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; # Stack now has: [<span class="m_-4722256188258047040gmail-syntax9">arrayname</span>, <span class="m_-4722256188258047040gmail-syntax9">indexvalue</span>]<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <span class="m_-4722256188258047040gmail-syntax9">incr</span> ::<span class="m_-4722256188258047040gmail-syntax9">tokpos</span>&nbsp;&nbsp;&nbsp;&nbsp; <u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;append <span class="m_-4722256188258047040gmail-syntax9">opcodes</span> "<span class="m_-4722256188258047040gmail-syntax9">loadArrayStk</span>; "<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return $<span class="m_-4722256188258047040gmail-syntax9">opcodes</span><u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">&nbsp;&nbsp;&nbsp; }<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u>&nbsp;<u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u>&nbsp;<u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">In addition, there has indeed been some changes in the <span class="m_-4722256188258047040gmail-syntax9">bytecode</span>, land and <span class="m_-4722256188258047040gmail-syntax9">lor</span> are no longer supported in 9.0 although they work in 8.6.<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u>&nbsp;<u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">I had an AI generate some 117 test cases, which all pass on 8.6 and 111 on 9.x (the land/<span class="m_-4722256188258047040gmail-syntax9">lor</span> not being tested in 9.x).<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u>&nbsp;<u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">Colin, with your permission, I can post the code as a new file, with all the test cases, say on a repository at <span class="m_-4722256188258047040gmail-syntax9">github</span>.<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u>&nbsp;<u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">I think a new TIP is worth considering; one that promotes assemble to a supported form, with a compile and handle approach to avoid the time parsing the <span class="m_-4722256188258047040gmail-syntax9">ascii</span> byte code text. I think that this would be great for your = command, but also quite useful for others who might want to create their own little languages.<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u>&nbsp;<u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">By doing it this way, it remains pure <span class="m_-4722256188258047040gmail-syntax9">tcl</span>, and avoids all the problems with different systems and hardware that a binary extension would create. In the end, I believe your code can achieve performance parity with expr. Not only does it remove half the [expr {...}] baggage, but all the $'s too! So much easier on these old eyes.<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u>&nbsp;<u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">Regards,<u></u><u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u>&nbsp;<u></u></span></pre>
<pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">Eric<u></u><u></u></span></pre>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
<div>
<div>
<p class="MsoNormal">On Tue, Nov 4, 2025 at 1:06<span style="font-family:&quot;Arial&quot;,sans-serif"> </span>PM EricT &lt;<a href="mailto:tw...@gm..." target="_blank">tw...@gm...</a>&gt; wrote:<u></u><u></u></p>
</div>
<blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:currentcolor currentcolor currentcolor rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal">Hi Colin,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">Hmmm, why can't you do bareword on $a(b) as a(b) you just need to do an uplevel to see if a is a variable, if not, it would have to be a function. True?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">% tcl::unsupported::disassemble script {set a [expr {$b($c)}] }<br>
snip<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12pt">&nbsp; Command 2: "expr {$b($c)}..."<br>
&nbsp; &nbsp; (2) push1 1 # "b"<br>
&nbsp; &nbsp; (4) push1 2 # "c"<br>
&nbsp; &nbsp; (6) loadStk <br>
&nbsp; &nbsp; (7) loadArrayStk <br>
&nbsp; &nbsp; (8) tryCvtToNumeric <br>
&nbsp; &nbsp; (9) storeStk <br>
&nbsp; &nbsp; (10) done&nbsp;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">This doesn't look too much different from what you are producing.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">I think what's really needed here is a TIP that would open up the bytecode a bit so you don't need to use an unsupported&nbsp;command. And then maybe even have a new command to take the string byte code you are now producing and return a handle
 to a cached version that was probably equivalent to the existing bytecode. Then your cache array would be<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">&nbsp;set cache($exp) $handle<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">Instead of it having to parse the text, it could be as fast as bytecode. You'd likely be just as fast as expr, and safe as well, since you can't pass a string command in where the bareword is required:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12pt">&nbsp;% set x {[pwd]}<br>
[pwd]<br>
&nbsp;% = sqrt(x)<br>
exp= |sqrt(x)| code= |push ::tcl::mathfunc::sqrt; push x; loadStk; invokeStk 2; | ifexist: 0<br>
expected floating-point number but got "[pwd]"<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">I think you really have something here, perhaps this is the best answer yet to slay the expr dragon!&nbsp;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">Regards,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">Eric<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
<div>
<div>
<p class="MsoNormal">On Tue, Nov 4, 2025 at 6:52<span style="font-family:&quot;Arial&quot;,sans-serif"> </span>AM Colin Macleod via Tcl-Core &lt;<a href="mailto:tcl...@li..." target="_blank">tcl...@li...</a>&gt; wrote:<u></u><u></u></p>
</div>
<blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:currentcolor currentcolor currentcolor rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p>Hi Eric,<br>
<br>
That's very neat! <br>
<br>
Yes, a pure Tcl version could go into TclLib. I still think it may be worth trying a C implementation though.&nbsp; The work-around that's needed for array references [= 2* $a(b)] would defeat the caching, so it would be good to speed up the parsing if possible.&nbsp;
 Also I think your caching may be equivalent to doing byte-compilation, in which case it may make sense to use the framework which already exists for that.<br>
<br>
Colin.<u></u><u></u></p>
<div>
<p class="MsoNormal">On 04/11/2025 01:18, EricT wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<div>
<div>
<p class="MsoNormal">that is:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">if {[info exist ::cache($exp)]} { <u></u><u></u></p>
</div>
<p class="MsoNormal">&nbsp; &nbsp; tailcall ::tcl::unsupported::assemble $::cache($exp) <br>
} <u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">(hate gmail!)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
<div>
<div>
<p class="MsoNormal">On Mon, Nov 3, 2025 at 5:17<span style="font-family:&quot;Arial&quot;,sans-serif"> </span>PM EricT &lt;<a href="mailto:tw...@gm..." target="_blank">tw...@gm...</a>&gt; wrote:<u></u><u></u></p>
</div>
<blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:currentcolor currentcolor currentcolor rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal" style="margin-bottom:12pt">and silly of me, it should be:<br>
if {[info exist ::cache($exp)]} { <br>
&nbsp; &nbsp; tailcall ::tcl::unsupported::assemble $::cache($exp) <br>
} <u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
<div>
<div>
<p class="MsoNormal">On Mon, Nov 3, 2025 at 4:50<span style="font-family:&quot;Arial&quot;,sans-serif"> </span>PM EricT &lt;<a href="mailto:tw...@gm..." target="_blank">tw...@gm...</a>&gt; wrote:<u></u><u></u></p>
</div>
<blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:currentcolor currentcolor currentcolor rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal">With a debug line back in plus the tailcall:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">proc = args {<u></u><u></u></p>
</div>
<p class="MsoNormal">&nbsp; &nbsp; set exp [join $args]<br>
&nbsp; &nbsp; if { [info exist ::cache($exp)] } {<br>
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;return [tailcall &nbsp;::tcl::unsupported::assemble $::cache($exp)]<br>
&nbsp; &nbsp; }<br>
&nbsp; &nbsp; set tokens [tokenise $exp]<br>
&nbsp; &nbsp; deb1 "TOKENS = '$tokens'"<br>
&nbsp; &nbsp; set code [compile $tokens]<br>
&nbsp; &nbsp; deb1 "GENERATED CODE:\n$code\n" <br>
&nbsp; &nbsp; puts "exp= |$exp| code= |$code| ifexist: [info exist ::cache($exp)]"<br>
&nbsp; &nbsp; set ::cache($exp) $code<br>
&nbsp; &nbsp; uplevel [list ::tcl::unsupported::assemble $code]<br>
}<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12pt">% set a 5<br>
5<br>
&nbsp;% set b 10<br>
10<br>
&nbsp;% = a + b<br>
exp= |a + b| code= |push a; loadStk; push b; loadStk; add; | ifexist: 0<br>
15<br>
% = a + b<br>
15<br>
<br>
% time {= a + b} 1000<br>
1.73 microseconds per iteration<br>
<br>
<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Faster still!<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">I thought the uplevel was needed to be able to get the local variables, seems not.<br>
<br>
&nbsp;% proc foo arg {set a 5; set b 10; set c [= a+b+arg]}<br>
&nbsp;% foo 5<br>
exp= |a+b+arg| code= |push a; loadStk; push b; loadStk; add; push arg; loadStk; add; | ifexist: 0<br>
20<br>
% foo 5<br>
20<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">&nbsp;% proc foo arg {global xxx; set a 5; set b 10; set c [= a+b+arg+xxx]}<br>
<br>
&nbsp;% set xxx 100<br>
100<br>
&nbsp;% foo 200<br>
315<br>
&nbsp;% time {foo 200} 10000<br>
2.1775 microseconds per iteration<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">&nbsp;% parray cache<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12pt">cache(a + b) &nbsp; &nbsp; &nbsp; = push a; loadStk; push b; loadStk; add;&nbsp;<br>
cache(a+b+arg) &nbsp; &nbsp; = push a; loadStk; push b; loadStk; add; push arg; loadStk; add;
<br>
cache(a+b+arg+xxx) = push a; loadStk; push b; loadStk; add; push arg; loadStk; add; push xxx; loadStk; add;
<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">Very Impressive, great job Colin! Great catch Don!<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">Eric<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
<div>
<div>
<p class="MsoNormal">On Mon, Nov 3, 2025 at 4:22<span style="font-family:&quot;Arial&quot;,sans-serif"> </span>PM Donald Porter via Tcl-Core &lt;<a href="mailto:tcl...@li..." target="_blank">tcl...@li...</a>&gt; wrote:<u></u><u></u></p>
</div>
<blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:currentcolor currentcolor currentcolor rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal">Check what effect replacing [uplevel] with [tailcall] has.<u></u><u></u></p>
<div>
<p class="MsoNormal"><br>
<br>
<u></u><u></u></p>
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<div>
<p class="MsoNormal">On Nov 3, 2025, at 7:13<span style="font-family:&quot;Arial&quot;,sans-serif"> </span>PM, EricT &lt;<a href="mailto:tw...@gm..." target="_blank">tw...@gm...</a>&gt; wrote:<u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
<div>
<div>
<p class="MsoNormal">Subject: Your bytecode expression evaluator - impressive results with caching!<br>
<br>
Hey Colin:<br>
<br>
I took a look at your bytecode-based expression evaluator and was intrigued by the approach. I made a small modification to add caching and the results are really impressive. Here's what I changed:<br>
<br>
proc = args {<br>
&nbsp; &nbsp; set exp [join $args]<br>
&nbsp; &nbsp; if {[info exist ::cache($exp)]} {<br>
&nbsp; &nbsp; &nbsp; &nbsp; return [uplevel [list ::tcl::unsupported::assemble $::cache($exp)]]<br>
&nbsp; &nbsp; }<br>
&nbsp; &nbsp; set tokens [tokenise $exp]<br>
&nbsp; &nbsp; deb1 "TOKENS = '$tokens'"<br>
&nbsp; &nbsp; set code [compile $tokens]<br>
&nbsp; &nbsp; deb1 "GENERATED CODE:\n$code\n" <br>
&nbsp; &nbsp; set ::cache($exp) $code<br>
&nbsp; &nbsp; uplevel [list ::tcl::unsupported::assemble $code]<br>
}<br>
<br>
The cache is just a simple array lookup - one line to store, one line to retrieve. But the performance impact is huge:<br>
<br>
Performance Tests<br>
<br>
Without caching<br>
% time {= 1 + 2} 1000<br>
24.937 microseconds per iteration<br>
<br>
With caching<br>
% time {= 1 + 2} 1000<br>
1.8 microseconds per iteration<br>
<br>
That's a 13x speedup! The tokenize and parse steps were eating about 92% of the execution time.<br>
<br>
The Real Magic: Bare Variables + Caching<br>
<br>
What really impressed me is how well your bare variable feature synergizes with caching:<br>
<br>
% set a 5<br>
5<br>
% set b 6 &nbsp;<br>
6<br>
% = a + b<br>
11<br>
% time {= a + b} 1000<br>
2.079 microseconds per iteration<br>
<br>
Now change the variable values<br>
% set a 10<br>
10<br>
% = a + b<br>
16<br>
% time {= a + b} 1000<br>
2.188 microseconds per iteration<br>
<br>
The cache entry stays valid even when the variable values change! Why? Because the bytecode stores variable names, not values:<br>
<br>
push a; loadStk; push b; loadStk; add;<br>
<br>
The loadStk instruction does runtime lookup, so:<br>
- Cache key is stable: "a + b"<br>
- Works for any values of a and b<br>
- One cache entry handles all value combinations<br>
<br>
Compare this to if we used $-substitution:<br>
<br>
= $a + $b &nbsp; &nbsp;# With a=5, b=6 becomes "5 + 6"<br>
= $a + $b &nbsp; &nbsp;# With a=10, b=6 becomes "10 + 6" - different cache key!<br>
<br>
Every value change would create a new cache entry or worse, a cache miss.<br>
<br>
Comparison to Other Approaches<br>
<br>
Tcl's expr: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;about 0.40 microseconds<br>
Direct C evaluator: &nbsp; &nbsp; &nbsp;about 0.53 microseconds &nbsp;<br>
Your cached approach: &nbsp; &nbsp;about 1.80 microseconds<br>
Your uncached approach: &nbsp;about 24.9 microseconds<br>
<br>
With caching, you're only 3-4x slower than a direct C evaluator.<br>
<br>
<br>
My Assessment<br>
<br>
Your design is excellent. The bare variable feature isn't just syntax sugar - it's essential for good cache performance. The synergy between:<br>
<br>
1. Bare variables leading to stable cache keys<br>
2. Runtime lookup keeping cache hot<br>
3. Simple caching providing dramatic speedup<br>
<br>
makes this really elegant.<br>
<br>
My recommendation: Keep it in Tcl! The implementation is clean, performance is excellent (1.8 microseconds is plenty fast), and converting to C would add significant complexity for minimal gain (maybe getting to about 1.0 microseconds).<br>
<br>
The Tcl prototype with caching is actually the right solution here. Sometimes the prototype IS the product!<br>
<br>
Excellent work on this. The bytecode approach really shines with caching enabled.<u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
<div>
<div>
<p class="MsoNormal">On Sun, Nov 2, 2025 at 10:14<span style="font-family:&quot;Arial&quot;,sans-serif"> </span>AM Colin Macleod via Tcl-Core &lt;<a href="mailto:tcl...@li..." target="_blank">tcl...@li...</a>&gt; wrote:<u></u><u></u></p>
</div>
<blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:currentcolor currentcolor currentcolor rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p>Hi again,<u></u><u></u></p>
<p>I've now made a slightly more serious prototype, see <a href="https://cmacleod.me.uk/tcl/expr_ng" target="_blank">
https://cmacleod.me.uk/tcl/expr_ng</a><u></u><u></u></p>
<p>This is a modified version of the prototype I wrote for tip 676. It's still in Tcl, but doesn't use `expr`.&nbsp; It tokenises and parses the input, then generates TAL bytecode and uses ::tcl::unsupported::assemble to run that.&nbsp; A few examples:<u></u><u></u></p>
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<p><span style="color:blue">(bin) 100 % set a [= 3.0/4]<br>
0.75<br>
(bin) 101 % set b [= sin(a*10)]<br>
0.9379999767747389<br>
(bin) 102 % set c [= (b-a)*100]<br>
18.79999767747389<br>
(bin) 103 % namespace eval nn {set d [= 10**3]}<br>
1000<br>
(bin) 104 % set e [= a?nn::d:b]<br>
1000<br>
(bin) 105 % = {3 + [pwd]}<br>
Calc: expected start of expression but found '[pwd]'<br>
(bin) 106 % = {3 + $q}<br>
Calc: expected start of expression but found '$q'<br>
(bin) 107 % = sin (12)<br>
-0.5365729180004349</span><u></u><u></u></p>
<p><span style="color:blue">(bin) 108 % array set rr {one 1 two 2 three 3}<br>
(bin) 110 % = a * rr(two)<br>
Calc: expected operator but found '('<br>
(bin) 111 % = a * $rr(two)<br>
1.5</span><u></u><u></u></p>
</blockquote>
<p>- You can use $ to get an array value substituted before the `=` code sees the expression.<u></u><u></u></p>
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<p><span style="color:blue">(bin) 112 % string repeat ! [= nn::d / 15]<br>
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!</span><u></u><u></u></p>
</blockquote>
<p>Colin.<u></u><u></u></p>
<div>
<p class="MsoNormal">On 02/11/2025 09:04, Donal Fellows wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<div>
<p class="MsoNormal">Doing the job properly would definitely involve changing the expression parser, with my suggested fix being to turn all bare words not otherwise recognised as constants or in positions that look like function calls (it's a parser with some
 lookahead) into simple variable reads (NB: C resolves such ambiguities within itself differently, but that's one of the nastiest parts of the language). We would need to retain $ support for resolving ambiguity (e.g., array reads vs function calls; you can't
 safely inspect the interpreter to resolve it at the time of compiling the expression due to traces and unknown handlers) as well as compatibility, but that's doable as it is a change only in cases that are currently errors.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">Adding assignment is quite a bit trickier, as that needs a new major syntax class to describe the left side of the assignment. I suggest omitting that from consideration at this stage.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">Donal.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">-------- Original message --------<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">From: Colin Macleod via Tcl-Core <a href="mailto:tcl...@li..." target="_blank">
&lt;tcl...@li...&gt;</a> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Date: 02/11/2025 08:13 (GMT+00:00) <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">To: Pietro Cerutti <a href="mailto:ga...@ga..." target="_blank">
&lt;ga...@ga...&gt;</a> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Cc: <a href="mailto:tcl...@li..." target="_blank">
tcl...@li...</a>, <a href="mailto:av...@lo..." target="_blank">
av...@lo...</a> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Subject: Re: [TCLCORE] Fwd: TIP 672 Implementation Complete - Ready for Sponsorship
<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<p>Indeed, this toy implementation doesn't handle that:<u></u><u></u></p>
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<p><span style="color:blue">% = sin (12)<br>
can't read "sin": no such variable</span><u></u><u></u></p>
</blockquote>
<p>I'm not sure that's serious, but it could be fixed in a C implementation.<u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal">_______________________________________________<br>
Tcl-Core mailing list<br>
<a href="mailto:Tcl...@li..." target="_blank">Tcl...@li...</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/tcl-core" target="_blank">https://lists.sourceforge.net/lists/listinfo/tcl-core</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal">_______________________________________________<br>
Tcl-Core mailing list<br>
<a href="mailto:Tcl...@li..." target="_blank">Tcl...@li...</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/tcl-core" target="_blank">https://lists.sourceforge.net/lists/listinfo/tcl-core</a><u></u><u></u></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<p class="MsoNormal">_______________________________________________<br>
Tcl-Core mailing list<br>
<a href="mailto:Tcl...@li..." target="_blank">Tcl...@li...</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/tcl-core" target="_blank">https://lists.sourceforge.net/lists/listinfo/tcl-core</a><u></u><u></u></p>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-bottom:12pt"><u></u>&nbsp;<u></u></p>
<pre>_______________________________________________<u></u><u></u></pre>
<pre>Tcl-Core mailing list<u></u><u></u></pre>
<pre><a href="mailto:Tcl...@li..." target="_blank">Tcl...@li...</a><u></u><u></u></pre>
<pre><a href="https://lists.sourceforge.net/lists/listinfo/tcl-core" target="_blank">https://lists.sourceforge.net/lists/listinfo/tcl-core</a><u></u><u></u></pre>
</blockquote>
</div>
<p class="MsoNormal">_______________________________________________<br>
Tcl-Core mailing list<br>
<a href="mailto:Tcl...@li..." target="_blank">Tcl...@li...</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/tcl-core" target="_blank">https://lists.sourceforge.net/lists/listinfo/tcl-core</a><u></u><u></u></p>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>

_______________________________________________<br>
Tcl-Core mailing list<br>
<a href="mailto:Tcl...@li..." target="_blank">Tcl...@li...</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/tcl-core" rel="noreferrer" target="_blank">https://lists.sourceforge.net/lists/listinfo/tcl-core</a><br>
</div></blockquote></div>
<span>_______________________________________________</span><br><span>Tcl-Core mailing list</span><br><span>Tcl...@li...</span><br><span>https://lists.sourceforge.net/lists/listinfo/tcl-core</span><br></div></blockquote></div></div></body></html>

Re: [TCLCORE] CFV: TIP 733, Add accessibility/screen reader support to Tk

[Steve L. <st...@di...>]

TIP #733: YES
On 2 Nov 2025 at 8:37 AM +0800, Kevin Walzer <kw...@co...>, wrote:
> Hi all,
>
> I've had no additional feedback on TIP 733 in the past week, and I
> believe the tka11y branch is ready for merging, so I'd like to call for
> a TCT vote.
>
> Let's have votes in by [clock format 1762559940 -gmt 1].
>
> --Kevin
>

Re: [TCLCORE] [Ext] Re: [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)

[EricT <tw...@gm...>]

Hi Rene,

I was experimenting with your TIP 674 prototype and realized both your
let command and Colin's expression evaluator share something
important: they could both be implemented in pure Tcl if
tcl::unsupported::assemble were promoted to supported status with
handle-based caching.

This would avoid another round of "expr wars" where lack of consensus
leaves us with nothing. Instead of debating whose syntax is better,
we'd each have the power to create our own Design-Specific Language.
With bytecode caching, pure Tcl implementations achieve C extension
performance - Colin's evaluator runs at 1.8 microseconds, nearly as
fast as expr.

Why does this matter for your TIP? If let performs just as well in Tcl
as in C, why require a C implementation at all? C extensions take
weeks to develop, need binaries for every platform/architecture
combination, and ultimately call the same bytecode infrastructure that
assemble uses. A supported bytecode API would make both proposals -
and many others - viable without C.

Perhaps we should advocate together for making the bytecode
compilation infrastructure officially supported, rather than competing
for TIP approval on individual commands?

Best,
Eric



On Wed, Nov 5, 2025 at 11:31 PM Zaumseil René via Tcl-Core <
tcl...@li...> wrote:

> Hello
>
>
>
> I also like the = approach as a new expr command without need of $ for
> variables.
>
> But I'm not sure about the currently proposed syntax.
>
> Is it "= 1 + 2" or "= 1+2" (with or without spaces)?
>
> If it is the later then I would propose to use some of the syntax of the
> "let" from tip 674.
>
>
>
> Some syntax ideas:
>
> "= 1+2" return 3, simple case
>
> "= a 1+2 b 2+4 c a+b" return {3 6 9} and set a,b,c
>
> "= 1+2 = 2+4" return {3 6}
>
> And may be some others…
>
>
>
> Regards
>
> rene
>
> *Von:* EricT <tw...@gm...>
> *Gesendet:* Mittwoch, 5. November 2025 23:22
> *An:* Colin Macleod <col...@ya...>;
> tcl...@li...
> *Betreff:* [Ext] Re: [TCLCORE] [=] for concise expressions (was Re: TIP
> 672 Implementation Complete - Ready for Sponsorship)
>
>
>
> Hi Colin,
>
>
>
> I've successfully modified your amazing code to handle arrays. In doing so, I also found 2 other issues, one is with your Boolean check, the other with your function name check, both because of [string is] issues.
>
>
>
> - Boolean check: `$token eq "false" || $token eq "true"` (was `[string is boolean $token]` - treated 'f','n', 't', 'y', etc. variables as boolean false, no, true, yes, ...)
>
>
>
> - Function check: `[regexp {^[[:alpha:]]} $token]` (was `[string is alpha $token]` - broke log10, atan2)
>
>
>
>
>
> here's the code for arrays:
>
>
>
>     # Function call or array reference?
>
>     set nexttok [lindex $::tokens $::tokpos]
>
>     if {$nexttok eq "(" && [regexp {^[[:alpha:]]} $token]} {
>
>         set fun [namespace which tcl::mathfunc::$token]
>
>         if {$fun ne {}} {
>
>             # It's a function
>
>             incr ::tokpos
>
>             set opcodes "push $fun; "
>
>             append opcodes [parseFuncArgs]
>
>             return $opcodes
>
>         } else {
>
>             # Not a function, assume array reference
>
>             incr ::tokpos
>
>             set opcodes "push $token; "
>
>             # Parse the index expression - leaves VALUE on stack
>
>             append opcodes [parse 0]
>
>             # Expect closing paren
>
>             set closing [lindex $::tokens $::tokpos]
>
>             if {$closing ne ")"} {
>
>                 error "Calc: expected ')' but found '$closing'"
>
>             }
>
>             # Stack now has: [arrayname, indexvalue]
>
>             incr ::tokpos
>
>             append opcodes "loadArrayStk; "
>
>             return $opcodes
>
>         }
>
>     }
>
>
>
>
>
> In addition, there has indeed been some changes in the bytecode, land and lor are no longer supported in 9.0 although they work in 8.6.
>
>
>
> I had an AI generate some 117 test cases, which all pass on 8.6 and 111 on 9.x (the land/lor not being tested in 9.x).
>
>
>
> Colin, with your permission, I can post the code as a new file, with all the test cases, say on a repository at github.
>
>
>
> I think a new TIP is worth considering; one that promotes assemble to a supported form, with a compile and handle approach to avoid the time parsing the ascii byte code text. I think that this would be great for your = command, but also quite useful for others who might want to create their own little languages.
>
>
>
> By doing it this way, it remains pure tcl, and avoids all the problems with different systems and hardware that a binary extension would create. In the end, I believe your code can achieve performance parity with expr. Not only does it remove half the [expr {...}] baggage, but all the $'s too! So much easier on these old eyes.
>
>
>
> Regards,
>
>
>
> Eric
>
>
>
>
>
> On Tue, Nov 4, 2025 at 1:06 PM EricT <tw...@gm...> wrote:
>
> Hi Colin,
>
>
>
> Hmmm, why can't you do bareword on $a(b) as a(b) you just need to do an
> uplevel to see if a is a variable, if not, it would have to be a function.
> True?
>
>
>
> % tcl::unsupported::disassemble script {set a [expr {$b($c)}] }
> snip
>
>   Command 2: "expr {$b($c)}..."
>     (2) push1 1 # "b"
>     (4) push1 2 # "c"
>     (6) loadStk
>     (7) loadArrayStk
>     (8) tryCvtToNumeric
>     (9) storeStk
>     (10) done
>
> This doesn't look too much different from what you are producing.
>
>
>
> I think what's really needed here is a TIP that would open up the bytecode
> a bit so you don't need to use an unsupported command. And then maybe even
> have a new command to take the string byte code you are now producing and
> return a handle to a cached version that was probably equivalent to the
> existing bytecode. Then your cache array would be
>
>
>
>  set cache($exp) $handle
>
>
>
> Instead of it having to parse the text, it could be as fast as bytecode.
> You'd likely be just as fast as expr, and safe as well, since you can't
> pass a string command in where the bareword is required:
>
>
>
>  % set x {[pwd]}
> [pwd]
>  % = sqrt(x)
> exp= |sqrt(x)| code= |push ::tcl::mathfunc::sqrt; push x; loadStk;
> invokeStk 2; | ifexist: 0
> expected floating-point number but got "[pwd]"
>
> I think you really have something here, perhaps this is the best answer
> yet to slay the expr dragon!
>
>
>
> Regards,
>
>
>
> Eric
>
>
>
>
>
> On Tue, Nov 4, 2025 at 6:52 AM Colin Macleod via Tcl-Core <
> tcl...@li...> wrote:
>
> Hi Eric,
>
> That's very neat!
>
> Yes, a pure Tcl version could go into TclLib. I still think it may be
> worth trying a C implementation though.  The work-around that's needed for
> array references [= 2* $a(b)] would defeat the caching, so it would be good
> to speed up the parsing if possible.  Also I think your caching may be
> equivalent to doing byte-compilation, in which case it may make sense to
> use the framework which already exists for that.
>
> Colin.
>
> On 04/11/2025 01:18, EricT wrote:
>
> that is:
>
>
>
> if {[info exist ::cache($exp)]} {
>
>     tailcall ::tcl::unsupported::assemble $::cache($exp)
> }
>
>
>
> (hate gmail!)
>
>
>
>
>
> On Mon, Nov 3, 2025 at 5:17 PM EricT <tw...@gm...> wrote:
>
> and silly of me, it should be:
> if {[info exist ::cache($exp)]} {
>     tailcall ::tcl::unsupported::assemble $::cache($exp)
> }
>
>
>
> On Mon, Nov 3, 2025 at 4:50 PM EricT <tw...@gm...> wrote:
>
> With a debug line back in plus the tailcall:
>
>
>
> proc = args {
>
>     set exp [join $args]
>     if { [info exist ::cache($exp)] } {
>          return [tailcall  ::tcl::unsupported::assemble $::cache($exp)]
>     }
>     set tokens [tokenise $exp]
>     deb1 "TOKENS = '$tokens'"
>     set code [compile $tokens]
>     deb1 "GENERATED CODE:\n$code\n"
>     puts "exp= |$exp| code= |$code| ifexist: [info exist ::cache($exp)]"
>     set ::cache($exp) $code
>     uplevel [list ::tcl::unsupported::assemble $code]
> }
>
>
>
> % set a 5
> 5
>  % set b 10
> 10
>  % = a + b
> exp= |a + b| code= |push a; loadStk; push b; loadStk; add; | ifexist: 0
> 15
> % = a + b
> 15
>
> % time {= a + b} 1000
> 1.73 microseconds per iteration
>
> Faster still!
>
>
>
> I thought the uplevel was needed to be able to get the local variables,
> seems not.
>
>  % proc foo arg {set a 5; set b 10; set c [= a+b+arg]}
>  % foo 5
> exp= |a+b+arg| code= |push a; loadStk; push b; loadStk; add; push arg;
> loadStk; add; | ifexist: 0
> 20
> % foo 5
> 20
>
>
>
>  % proc foo arg {global xxx; set a 5; set b 10; set c [= a+b+arg+xxx]}
>
>  % set xxx 100
> 100
>  % foo 200
> 315
>  % time {foo 200} 10000
> 2.1775 microseconds per iteration
>
>
>
>  % parray cache
>
> cache(a + b)       = push a; loadStk; push b; loadStk; add;
> cache(a+b+arg)     = push a; loadStk; push b; loadStk; add; push arg;
> loadStk; add;
> cache(a+b+arg+xxx) = push a; loadStk; push b; loadStk; add; push arg;
> loadStk; add; push xxx; loadStk; add;
>
>
>
> Very Impressive, great job Colin! Great catch Don!
>
>
>
> Eric
>
>
>
>
>
>
>
>
>
> On Mon, Nov 3, 2025 at 4:22 PM Donald Porter via Tcl-Core <
> tcl...@li...> wrote:
>
> Check what effect replacing [uplevel] with [tailcall] has.
>
>
>
> On Nov 3, 2025, at 7:13 PM, EricT <tw...@gm...> wrote:
>
>
>
> Subject: Your bytecode expression evaluator - impressive results with
> caching!
>
> Hey Colin:
>
> I took a look at your bytecode-based expression evaluator and was
> intrigued by the approach. I made a small modification to add caching and
> the results are really impressive. Here's what I changed:
>
> proc = args {
>     set exp [join $args]
>     if {[info exist ::cache($exp)]} {
>         return [uplevel [list ::tcl::unsupported::assemble $::cache($exp)]]
>     }
>     set tokens [tokenise $exp]
>     deb1 "TOKENS = '$tokens'"
>     set code [compile $tokens]
>     deb1 "GENERATED CODE:\n$code\n"
>     set ::cache($exp) $code
>     uplevel [list ::tcl::unsupported::assemble $code]
> }
>
> The cache is just a simple array lookup - one line to store, one line to
> retrieve. But the performance impact is huge:
>
> Performance Tests
>
> Without caching
> % time {= 1 + 2} 1000
> 24.937 microseconds per iteration
>
> With caching
> % time {= 1 + 2} 1000
> 1.8 microseconds per iteration
>
> That's a 13x speedup! The tokenize and parse steps were eating about 92%
> of the execution time.
>
> The Real Magic: Bare Variables + Caching
>
> What really impressed me is how well your bare variable feature synergizes
> with caching:
>
> % set a 5
> 5
> % set b 6
> 6
> % = a + b
> 11
> % time {= a + b} 1000
> 2.079 microseconds per iteration
>
> Now change the variable values
> % set a 10
> 10
> % = a + b
> 16
> % time {= a + b} 1000
> 2.188 microseconds per iteration
>
> The cache entry stays valid even when the variable values change! Why?
> Because the bytecode stores variable names, not values:
>
> push a; loadStk; push b; loadStk; add;
>
> The loadStk instruction does runtime lookup, so:
> - Cache key is stable: "a + b"
> - Works for any values of a and b
> - One cache entry handles all value combinations
>
> Compare this to if we used $-substitution:
>
> = $a + $b    # With a=5, b=6 becomes "5 + 6"
> = $a + $b    # With a=10, b=6 becomes "10 + 6" - different cache key!
>
> Every value change would create a new cache entry or worse, a cache miss.
>
> Comparison to Other Approaches
>
> Tcl's expr:              about 0.40 microseconds
> Direct C evaluator:      about 0.53 microseconds
> Your cached approach:    about 1.80 microseconds
> Your uncached approach:  about 24.9 microseconds
>
> With caching, you're only 3-4x slower than a direct C evaluator.
>
>
> My Assessment
>
> Your design is excellent. The bare variable feature isn't just syntax
> sugar - it's essential for good cache performance. The synergy between:
>
> 1. Bare variables leading to stable cache keys
> 2. Runtime lookup keeping cache hot
> 3. Simple caching providing dramatic speedup
>
> makes this really elegant.
>
> My recommendation: Keep it in Tcl! The implementation is clean,
> performance is excellent (1.8 microseconds is plenty fast), and converting
> to C would add significant complexity for minimal gain (maybe getting to
> about 1.0 microseconds).
>
> The Tcl prototype with caching is actually the right solution here.
> Sometimes the prototype IS the product!
>
> Excellent work on this. The bytecode approach really shines with caching
> enabled.
>
>
>
> On Sun, Nov 2, 2025 at 10:14 AM Colin Macleod via Tcl-Core <
> tcl...@li...> wrote:
>
> Hi again,
>
> I've now made a slightly more serious prototype, see
> https://cmacleod.me.uk/tcl/expr_ng
>
> This is a modified version of the prototype I wrote for tip 676. It's
> still in Tcl, but doesn't use `expr`.  It tokenises and parses the input,
> then generates TAL bytecode and uses ::tcl::unsupported::assemble to run
> that.  A few examples:
>
> (bin) 100 % set a [= 3.0/4]
> 0.75
> (bin) 101 % set b [= sin(a*10)]
> 0.9379999767747389
> (bin) 102 % set c [= (b-a)*100]
> 18.79999767747389
> (bin) 103 % namespace eval nn {set d [= 10**3]}
> 1000
> (bin) 104 % set e [= a?nn::d:b]
> 1000
> (bin) 105 % = {3 + [pwd]}
> Calc: expected start of expression but found '[pwd]'
> (bin) 106 % = {3 + $q}
> Calc: expected start of expression but found '$q'
> (bin) 107 % = sin (12)
> -0.5365729180004349
>
> (bin) 108 % array set rr {one 1 two 2 three 3}
> (bin) 110 % = a * rr(two)
> Calc: expected operator but found '('
> (bin) 111 % = a * $rr(two)
> 1.5
>
> - You can use $ to get an array value substituted before the `=` code sees
> the expression.
>
> (bin) 112 % string repeat ! [= nn::d / 15]
> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>
> Colin.
>
> On 02/11/2025 09:04, Donal Fellows wrote:
>
> Doing the job properly would definitely involve changing the expression
> parser, with my suggested fix being to turn all bare words not otherwise
> recognised as constants or in positions that look like function calls (it's
> a parser with some lookahead) into simple variable reads (NB: C resolves
> such ambiguities within itself differently, but that's one of the nastiest
> parts of the language). We would need to retain $ support for resolving
> ambiguity (e.g., array reads vs function calls; you can't safely inspect
> the interpreter to resolve it at the time of compiling the expression due
> to traces and unknown handlers) as well as compatibility, but that's doable
> as it is a change only in cases that are currently errors.
>
>
>
> Adding assignment is quite a bit trickier, as that needs a new major
> syntax class to describe the left side of the assignment. I suggest
> omitting that from consideration at this stage.
>
>
>
> Donal.
>
>
>
> -------- Original message --------
>
> From: Colin Macleod via Tcl-Core <tcl...@li...>
> <tcl...@li...>
>
> Date: 02/11/2025 08:13 (GMT+00:00)
>
> To: Pietro Cerutti <ga...@ga...> <ga...@ga...>
>
> Cc: tcl...@li..., av...@lo...
>
> Subject: Re: [TCLCORE] Fwd: TIP 672 Implementation Complete - Ready for
> Sponsorship
>
>
>
> Indeed, this toy implementation doesn't handle that:
>
> % = sin (12)
> can't read "sin": no such variable
>
> I'm not sure that's serious, but it could be fixed in a C implementation.
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>
>
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>
>
>
> _______________________________________________
>
> Tcl-Core mailing list
>
> Tcl...@li...
>
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>

Re: [TCLCORE] [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)

[EricT <tw...@gm...>]

Colin,

Thanks for the reply. I understand your consistency concerns -
predictability is important for language design.

I think it's important to recognize that this is a new "little language"
with its own grammar and semantics, separate from Tcl's expression syntax.
Because you're designing it fresh, you have the freedom to make deliberate
choices. For instance, reserving true and false as keywords makes perfect
sense in this language. Similarly, you can define how the language handles
the name(arg) pattern.

Regarding Perl - interestingly, both Perl and Tcl inherited the $ notation
from shell scripting. Dropping it in this new language is actually moving
away from Perl-like syntax, not towards it. But it's much more important
than just convenience that dropping it serves, it is the key to caching.

For example,

array set data {1 100 2 200}
set idx 1
= $data($idx) * 2      ;# Becomes: = 100 * 2, cache key: "100 * 2"
set idx 2
= $data($idx) * 2      ;# Becomes: = 200 * 2, cache key: "200 * 2" - new
entry!

Every value change creates a new cache entry and defeats the ability to
compile expressions, similar to unbraced expr. My factorial test case
(iterative loop to 50) showed 49 cache hits on one entry with bare
variables; with $-syntax, those would become cache misses.

When I approached the array vs. function choice, I was actually quite
surprised by your choice of testing for functions from the tcl::mathfunc::
namespace, which is much better than my first thought of using uplevel and
checking for array exists, though that is still an option. Either can work,
but by checking for mathfunc, you're allowing for custom functions. In
fact, here's how I added a fibonacci function to = with almost no effort:

package require math
proc tcl::mathfunc::fibonacci {n} {
    return [math::fibonacci $n]
}

= fibonacci(x) + fibonacci(y)

The namespace lookup is elegant and extensible. The ambiguity could be
documented (functions checked first, then arrays) or could be checked and
an error thrown, it's all up to the new language's designer.

Regards,

Eric

Here's where you can find it with the mods and tests:

https://github.com/rocketship88/colin-parser/tree/main

On Thu, Nov 6, 2025 at 1:37 AM Colin Macleod via Tcl-Core <
tcl...@li...> wrote:

> Hi Eric, thanks for your support.
>
> For the boolean check I think it's more consistent to disallow alphabetic
> forms entirely and only accept numeric zero or non-zero, which is what
> boolean operations and functions return anyway. An alphabetic string should
> always be treated as a variable reference, no exceptions to worry about.
>
> Similarly, i don't like the idea of  foo(bar)  sometimes being treated as
> a function call and sometimes as an array reference, depending on what
> definitions have been made elsewhere.  One should be able to tell what kind
> of thing it is just from the expression code, without searching for other
> definitions.  It is still possible to include an array reference by writing
> $foo(bar) so I would treat  foo(bar)  as a function call always, and fail
> if the function has not been defined.  Sacrificing consistency for minor
> convenience is the slippery slope that leads to Perl.  :-)
>
> You are welcome to post your modified version anywhere you like.
>
> Personally I still want to try a C implementation, but that will take me a
> few weeks.
>
> Best regards,
> Colin.
> On 05/11/2025 22:22, EricT wrote:
>
> Hi Colin,
>
> I've successfully modified your amazing code to handle arrays. In doing so, I also found 2 other issues, one is with your Boolean check, the other with your function name check, both because of [string is] issues.
>
> - Boolean check: `$token eq "false" || $token eq "true"` (was `[string is boolean $token]` - treated 'f','n', 't', 'y', etc. variables as boolean false, no, true, yes, ...)
>
> - Function check: `[regexp {^[[:alpha:]]} $token]` (was `[string is alpha $token]` - broke log10, atan2)
>
>
> here's the code for arrays:
>
>     # Function call or array reference?
>     set nexttok [lindex $::tokens $::tokpos]
>     if {$nexttok eq "(" && [regexp {^[[:alpha:]]} $token]} {
>         set fun [namespace which tcl::mathfunc::$token]
>         if {$fun ne {}} {
>             # It's a function
>             incr ::tokpos
>             set opcodes "push $fun; "
>             append opcodes [parseFuncArgs]
>             return $opcodes
>         } else {
>             # Not a function, assume array reference
>             incr ::tokpos
>             set opcodes "push $token; "
>             # Parse the index expression - leaves VALUE on stack
>             append opcodes [parse 0]
>             # Expect closing paren
>             set closing [lindex $::tokens $::tokpos]
>             if {$closing ne ")"} {
>                 error "Calc: expected ')' but found '$closing'"
>             }
>             # Stack now has: [arrayname, indexvalue]
>             incr ::tokpos
>             append opcodes "loadArrayStk; "
>             return $opcodes
>         }
>     }
>
>
> In addition, there has indeed been some changes in the bytecode, land and lor are no longer supported in 9.0 although they work in 8.6.
>
> I had an AI generate some 117 test cases, which all pass on 8.6 and 111 on 9.x (the land/lor not being tested in 9.x).
>
> Colin, with your permission, I can post the code as a new file, with all the test cases, say on a repository at github.
>
> I think a new TIP is worth considering; one that promotes assemble to a supported form, with a compile and handle approach to avoid the time parsing the ascii byte code text. I think that this would be great for your = command, but also quite useful for others who might want to create their own little languages.
>
> By doing it this way, it remains pure tcl, and avoids all the problems with different systems and hardware that a binary extension would create. In the end, I believe your code can achieve performance parity with expr. Not only does it remove half the [expr {...}] baggage, but all the $'s too! So much easier on these old eyes.
>
> Regards,
>
> Eric
>
>
>
> On Tue, Nov 4, 2025 at 1:06 PM EricT <tw...@gm...> wrote:
>
>> Hi Colin,
>>
>> Hmmm, why can't you do bareword on $a(b) as a(b) you just need to do an
>> uplevel to see if a is a variable, if not, it would have to be a function.
>> True?
>>
>> % tcl::unsupported::disassemble script {set a [expr {$b($c)}] }
>> snip
>>   Command 2: "expr {$b($c)}..."
>>     (2) push1 1 # "b"
>>     (4) push1 2 # "c"
>>     (6) loadStk
>>     (7) loadArrayStk
>>     (8) tryCvtToNumeric
>>     (9) storeStk
>>     (10) done
>>
>> This doesn't look too much different from what you are producing.
>>
>> I think what's really needed here is a TIP that would open up the
>> bytecode a bit so you don't need to use an unsupported command. And then
>> maybe even have a new command to take the string byte code you are now
>> producing and return a handle to a cached version that was probably
>> equivalent to the existing bytecode. Then your cache array would be
>>
>>  set cache($exp) $handle
>>
>> Instead of it having to parse the text, it could be as fast as bytecode.
>> You'd likely be just as fast as expr, and safe as well, since you can't
>> pass a string command in where the bareword is required:
>>
>>  % set x {[pwd]}
>> [pwd]
>>  % = sqrt(x)
>> exp= |sqrt(x)| code= |push ::tcl::mathfunc::sqrt; push x; loadStk;
>> invokeStk 2; | ifexist: 0
>> expected floating-point number but got "[pwd]"
>>
>> I think you really have something here, perhaps this is the best answer
>> yet to slay the expr dragon!
>>
>> Regards,
>>
>> Eric
>>
>>
>> On Tue, Nov 4, 2025 at 6:52 AM Colin Macleod via Tcl-Core <
>> tcl...@li...> wrote:
>>
>>> Hi Eric,
>>>
>>> That's very neat!
>>>
>>> Yes, a pure Tcl version could go into TclLib. I still think it may be
>>> worth trying a C implementation though.  The work-around that's needed for
>>> array references [= 2* $a(b)] would defeat the caching, so it would be good
>>> to speed up the parsing if possible.  Also I think your caching may be
>>> equivalent to doing byte-compilation, in which case it may make sense to
>>> use the framework which already exists for that.
>>>
>>> Colin.
>>> On 04/11/2025 01:18, EricT wrote:
>>>
>>> that is:
>>>
>>> if {[info exist ::cache($exp)]} {
>>>     tailcall ::tcl::unsupported::assemble $::cache($exp)
>>> }
>>>
>>> (hate gmail!)
>>>
>>>
>>> On Mon, Nov 3, 2025 at 5:17 PM EricT <tw...@gm...> wrote:
>>>
>>>> and silly of me, it should be:
>>>> if {[info exist ::cache($exp)]} {
>>>>     tailcall ::tcl::unsupported::assemble $::cache($exp)
>>>> }
>>>>
>>>>
>>>> On Mon, Nov 3, 2025 at 4:50 PM EricT <tw...@gm...> wrote:
>>>>
>>>>> With a debug line back in plus the tailcall:
>>>>>
>>>>> proc = args {
>>>>>     set exp [join $args]
>>>>>     if { [info exist ::cache($exp)] } {
>>>>>          return [tailcall  ::tcl::unsupported::assemble $::cache($exp)]
>>>>>     }
>>>>>     set tokens [tokenise $exp]
>>>>>     deb1 "TOKENS = '$tokens'"
>>>>>     set code [compile $tokens]
>>>>>     deb1 "GENERATED CODE:\n$code\n"
>>>>>     puts "exp= |$exp| code= |$code| ifexist: [info exist
>>>>> ::cache($exp)]"
>>>>>     set ::cache($exp) $code
>>>>>     uplevel [list ::tcl::unsupported::assemble $code]
>>>>> }
>>>>>
>>>>> % set a 5
>>>>> 5
>>>>>  % set b 10
>>>>> 10
>>>>>  % = a + b
>>>>> exp= |a + b| code= |push a; loadStk; push b; loadStk; add; | ifexist: 0
>>>>> 15
>>>>> % = a + b
>>>>> 15
>>>>>
>>>>> % time {= a + b} 1000
>>>>> 1.73 microseconds per iteration
>>>>>
>>>>>
>>>>> Faster still!
>>>>>
>>>>> I thought the uplevel was needed to be able to get the local
>>>>> variables, seems not.
>>>>>
>>>>>  % proc foo arg {set a 5; set b 10; set c [= a+b+arg]}
>>>>>  % foo 5
>>>>> exp= |a+b+arg| code= |push a; loadStk; push b; loadStk; add; push arg;
>>>>> loadStk; add; | ifexist: 0
>>>>> 20
>>>>> % foo 5
>>>>> 20
>>>>>
>>>>>  % proc foo arg {global xxx; set a 5; set b 10; set c [= a+b+arg+xxx]}
>>>>>
>>>>>  % set xxx 100
>>>>> 100
>>>>>  % foo 200
>>>>> 315
>>>>>  % time {foo 200} 10000
>>>>> 2.1775 microseconds per iteration
>>>>>
>>>>>  % parray cache
>>>>> cache(a + b)       = push a; loadStk; push b; loadStk; add;
>>>>> cache(a+b+arg)     = push a; loadStk; push b; loadStk; add; push arg;
>>>>> loadStk; add;
>>>>> cache(a+b+arg+xxx) = push a; loadStk; push b; loadStk; add; push arg;
>>>>> loadStk; add; push xxx; loadStk; add;
>>>>>
>>>>>
>>>>> Very Impressive, great job Colin! Great catch Don!
>>>>>
>>>>> Eric
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Nov 3, 2025 at 4:22 PM Donald Porter via Tcl-Core <
>>>>> tcl...@li...> wrote:
>>>>>
>>>>>> Check what effect replacing [uplevel] with [tailcall] has.
>>>>>>
>>>>>> On Nov 3, 2025, at 7:13 PM, EricT <tw...@gm...> wrote:
>>>>>>
>>>>>> Subject: Your bytecode expression evaluator - impressive results with
>>>>>> caching!
>>>>>>
>>>>>> Hey Colin:
>>>>>>
>>>>>> I took a look at your bytecode-based expression evaluator and was
>>>>>> intrigued by the approach. I made a small modification to add caching and
>>>>>> the results are really impressive. Here's what I changed:
>>>>>>
>>>>>> proc = args {
>>>>>>     set exp [join $args]
>>>>>>     if {[info exist ::cache($exp)]} {
>>>>>>         return [uplevel [list ::tcl::unsupported::assemble
>>>>>> $::cache($exp)]]
>>>>>>     }
>>>>>>     set tokens [tokenise $exp]
>>>>>>     deb1 "TOKENS = '$tokens'"
>>>>>>     set code [compile $tokens]
>>>>>>     deb1 "GENERATED CODE:\n$code\n"
>>>>>>     set ::cache($exp) $code
>>>>>>     uplevel [list ::tcl::unsupported::assemble $code]
>>>>>> }
>>>>>>
>>>>>> The cache is just a simple array lookup - one line to store, one line
>>>>>> to retrieve. But the performance impact is huge:
>>>>>>
>>>>>> Performance Tests
>>>>>>
>>>>>> Without caching
>>>>>> % time {= 1 + 2} 1000
>>>>>> 24.937 microseconds per iteration
>>>>>>
>>>>>> With caching
>>>>>> % time {= 1 + 2} 1000
>>>>>> 1.8 microseconds per iteration
>>>>>>
>>>>>> That's a 13x speedup! The tokenize and parse steps were eating about
>>>>>> 92% of the execution time.
>>>>>>
>>>>>> The Real Magic: Bare Variables + Caching
>>>>>>
>>>>>> What really impressed me is how well your bare variable feature
>>>>>> synergizes with caching:
>>>>>>
>>>>>> % set a 5
>>>>>> 5
>>>>>> % set b 6
>>>>>> 6
>>>>>> % = a + b
>>>>>> 11
>>>>>> % time {= a + b} 1000
>>>>>> 2.079 microseconds per iteration
>>>>>>
>>>>>> Now change the variable values
>>>>>> % set a 10
>>>>>> 10
>>>>>> % = a + b
>>>>>> 16
>>>>>> % time {= a + b} 1000
>>>>>> 2.188 microseconds per iteration
>>>>>>
>>>>>> The cache entry stays valid even when the variable values change!
>>>>>> Why? Because the bytecode stores variable names, not values:
>>>>>>
>>>>>> push a; loadStk; push b; loadStk; add;
>>>>>>
>>>>>> The loadStk instruction does runtime lookup, so:
>>>>>> - Cache key is stable: "a + b"
>>>>>> - Works for any values of a and b
>>>>>> - One cache entry handles all value combinations
>>>>>>
>>>>>> Compare this to if we used $-substitution:
>>>>>>
>>>>>> = $a + $b    # With a=5, b=6 becomes "5 + 6"
>>>>>> = $a + $b    # With a=10, b=6 becomes "10 + 6" - different cache key!
>>>>>>
>>>>>> Every value change would create a new cache entry or worse, a cache
>>>>>> miss.
>>>>>>
>>>>>> Comparison to Other Approaches
>>>>>>
>>>>>> Tcl's expr:              about 0.40 microseconds
>>>>>> Direct C evaluator:      about 0.53 microseconds
>>>>>> Your cached approach:    about 1.80 microseconds
>>>>>> Your uncached approach:  about 24.9 microseconds
>>>>>>
>>>>>> With caching, you're only 3-4x slower than a direct C evaluator.
>>>>>>
>>>>>>
>>>>>> My Assessment
>>>>>>
>>>>>> Your design is excellent. The bare variable feature isn't just syntax
>>>>>> sugar - it's essential for good cache performance. The synergy between:
>>>>>>
>>>>>> 1. Bare variables leading to stable cache keys
>>>>>> 2. Runtime lookup keeping cache hot
>>>>>> 3. Simple caching providing dramatic speedup
>>>>>>
>>>>>> makes this really elegant.
>>>>>>
>>>>>> My recommendation: Keep it in Tcl! The implementation is clean,
>>>>>> performance is excellent (1.8 microseconds is plenty fast), and converting
>>>>>> to C would add significant complexity for minimal gain (maybe getting to
>>>>>> about 1.0 microseconds).
>>>>>>
>>>>>> The Tcl prototype with caching is actually the right solution here.
>>>>>> Sometimes the prototype IS the product!
>>>>>>
>>>>>> Excellent work on this. The bytecode approach really shines with
>>>>>> caching enabled.
>>>>>>
>>>>>> On Sun, Nov 2, 2025 at 10:14 AM Colin Macleod via Tcl-Core <
>>>>>> tcl...@li...> wrote:
>>>>>>
>>>>>>> Hi again,
>>>>>>>
>>>>>>> I've now made a slightly more serious prototype, see
>>>>>>> https://cmacleod.me.uk/tcl/expr_ng
>>>>>>>
>>>>>>> This is a modified version of the prototype I wrote for tip 676.
>>>>>>> It's still in Tcl, but doesn't use `expr`.  It tokenises and parses the
>>>>>>> input, then generates TAL bytecode and uses ::tcl::unsupported::assemble to
>>>>>>> run that.  A few examples:
>>>>>>>
>>>>>>> (bin) 100 % set a [= 3.0/4]
>>>>>>> 0.75
>>>>>>> (bin) 101 % set b [= sin(a*10)]
>>>>>>> 0.9379999767747389
>>>>>>> (bin) 102 % set c [= (b-a)*100]
>>>>>>> 18.79999767747389
>>>>>>> (bin) 103 % namespace eval nn {set d [= 10**3]}
>>>>>>> 1000
>>>>>>> (bin) 104 % set e [= a?nn::d:b]
>>>>>>> 1000
>>>>>>> (bin) 105 % = {3 + [pwd]}
>>>>>>> Calc: expected start of expression but found '[pwd]'
>>>>>>> (bin) 106 % = {3 + $q}
>>>>>>> Calc: expected start of expression but found '$q'
>>>>>>> (bin) 107 % = sin (12)
>>>>>>> -0.5365729180004349
>>>>>>>
>>>>>>> (bin) 108 % array set rr {one 1 two 2 three 3}
>>>>>>> (bin) 110 % = a * rr(two)
>>>>>>> Calc: expected operator but found '('
>>>>>>> (bin) 111 % = a * $rr(two)
>>>>>>> 1.5
>>>>>>>
>>>>>>> - You can use $ to get an array value substituted before the `=`
>>>>>>> code sees the expression.
>>>>>>>
>>>>>>> (bin) 112 % string repeat ! [= nn::d / 15]
>>>>>>> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>>>>>>>
>>>>>>> Colin.
>>>>>>> On 02/11/2025 09:04, Donal Fellows wrote:
>>>>>>>
>>>>>>> Doing the job properly would definitely involve changing the
>>>>>>> expression parser, with my suggested fix being to turn all bare words not
>>>>>>> otherwise recognised as constants or in positions that look like function
>>>>>>> calls (it's a parser with some lookahead) into simple variable reads (NB: C
>>>>>>> resolves such ambiguities within itself differently, but that's one of the
>>>>>>> nastiest parts of the language). We would need to retain $ support for
>>>>>>> resolving ambiguity (e.g., array reads vs function calls; you can't safely
>>>>>>> inspect the interpreter to resolve it at the time of compiling the
>>>>>>> expression due to traces and unknown handlers) as well as compatibility,
>>>>>>> but that's doable as it is a change only in cases that are currently errors.
>>>>>>>
>>>>>>> Adding assignment is quite a bit trickier, as that needs a new major
>>>>>>> syntax class to describe the left side of the assignment. I suggest
>>>>>>> omitting that from consideration at this stage.
>>>>>>>
>>>>>>> Donal.
>>>>>>>
>>>>>>> -------- Original message --------
>>>>>>> From: Colin Macleod via Tcl-Core <tcl...@li...>
>>>>>>> <tcl...@li...>
>>>>>>> Date: 02/11/2025 08:13 (GMT+00:00)
>>>>>>> To: Pietro Cerutti <ga...@ga...> <ga...@ga...>
>>>>>>> Cc: tcl...@li..., av...@lo...
>>>>>>> Subject: Re: [TCLCORE] Fwd: TIP 672 Implementation Complete - Ready
>>>>>>> for Sponsorship
>>>>>>>
>>>>>>> Indeed, this toy implementation doesn't handle that:
>>>>>>>
>>>>>>> % = sin (12)
>>>>>>> can't read "sin": no such variable
>>>>>>>
>>>>>>> I'm not sure that's serious, but it could be fixed in a C
>>>>>>> implementation.
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Tcl-Core mailing list
>>>>>>> Tcl...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/tcl-core
>>>>>>>
>>>>>> _______________________________________________
>>>>>> Tcl-Core mailing list
>>>>>> Tcl...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/tcl-core
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Tcl-Core mailing list
>>>>>> Tcl...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/tcl-core
>>>>>>
>>>>>
>>>
>>> _______________________________________________
>>> Tcl-Core mailing lis...@li...://lists.sourceforge.net/lists/listinfo/tcl-core
>>>
>>> _______________________________________________
>>> Tcl-Core mailing list
>>> Tcl...@li...
>>> https://lists.sourceforge.net/lists/listinfo/tcl-core
>>>
>> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>

Re: [TCLCORE] CFV: TIP 733, Add accessibility/screen reader support to Tk

[<apn...@ya...>]

TIP 733 YES

Yahoo Mail: Search, organise, conquer 
 
  On Thu, 6 Nov 2025 at 9:03 pm, Marc Culler<cul...@gm...> wrote:   _______________________________________________
Tcl-Core mailing list
Tcl...@li...
https://lists.sourceforge.net/lists/listinfo/tcl-core
  

Re: [TCLCORE] Tcl/Tk 9.0.3 Release Candidate

[Pietro C. <ga...@ga...>]

On Nov 05 2025, 13:16 +0000, Jan Nijtmans <jan...@gm...> wrote:
>Now available at
>
>https://sourceforge.net/projects/tcl/files/Tcl/9.0.3/
>
>are RC0 candidate source code distribution pre-releases of Tcl/Tk 9.0.3
>
>This is the first candidate release leading to the release of Tcl/Tk
>9.0.3. Testing of builds
>and operations on multiple platforms is invited.  Any critical problem
>that should block
>the release should be reported immediately.

All good on FreeBSD, thanks!

-- 
Pietro Cerutti
I have pledged to give 10% of income to effective charities
and invite you to join me - https://givingwhatwecan.org

Re: [TCLCORE] CFV: TIP 733, Add accessibility/screen reader support to Tk

[Marc C. <cul...@gm...>]

TIP 733: YES

- Marc

On Sat, Nov 1, 2025 at 7:36 PM Kevin Walzer <kw...@co...> wrote:

> Hi all,
>
> I've had no additional feedback on TIP 733 in the past week, and I
> believe the tka11y branch is ready for merging, so I'd like to call for
> a TCT vote.
>
> Let's have votes in by [clock format 1762559940 -gmt 1].
>
> My vote: TIP 733: YES
>
> --Kevin
>
>
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>

Re: [TCLCORE] CFV: TIP 733, Add accessibility/screen reader support to Tk

[Jan N. <jan...@gm...>]

Op zo 2 nov 2025 om 01:36 schreef Kevin Walzer:
> Let's have votes in by [clock format 1762559940 -gmt 1].
>

TIP 733: YES

Thanks!
         Jan Nijtmans

Re: [TCLCORE] Tcl/Tk 9.0.3 Release Candidate

[Erik L. <el...@xs...>]

On 11/5/25 14:16, Jan Nijtmans wrote:

> ... Testing of builds
> and operations on multiple platforms is invited.

The native build for x84_64-linux, and the cross-build linux -> x86_64-mingw32
(using x86_64-w64-mingw32-gcc 9.2.0) proceeded to completion.

Erik Leunissen.
--

Re: [TCLCORE] [Ext] Re: [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)

[Colin M. <col...@ya...>]

Hi Rene,

I want to accept both forms, with or without spaces. I think assignments 
and multiple values could be future enhancements, but for clarity I 
would use `=` and `,` as explicit operators for these.

Colin.

On 06/11/2025 07:30, Zaumseil René wrote:
>
> Hello
>
> I also like the = approach as a new expr command without need of $ for 
> variables.
>
> But I'm not sure about the currently proposed syntax.
>
> Is it "= 1 + 2" or "= 1+2" (with or without spaces)?
>
> If it is the later then I would propose to use some of the syntax of 
> the "let" from tip 674.
>
> Some syntax ideas:
>
> "= 1+2" return 3, simple case
>
> "= a 1+2 b 2+4 c a+b" return {3 6 9} and set a,b,c
>
> "= 1+2 = 2+4" return {3 6}
>
> And may be some others…
>
> Regards
>
> rene
>
> *Von:*EricT <tw...@gm...>
> *Gesendet:* Mittwoch, 5. November 2025 23:22
> *An:* Colin Macleod <col...@ya...>; 
> tcl...@li...
> *Betreff:* [Ext] Re: [TCLCORE] [=] for concise expressions (was Re: 
> TIP 672 Implementation Complete - Ready for Sponsorship)
>
> Hi Colin,
> I've successfully modified your amazing code to handle arrays. In 
> doing so, I also found 2 other issues, one is with your Boolean check, 
> the other with your function name check, both because of [string is] 
> issues.
> - Boolean check: `$token eq "false" || $token eq "true"` (was `[string 
> is boolean $token]` - treated 'f','n', 't', 'y', etc. variables as 
> boolean false, no, true, yes, ...)
> - Function check: `[regexp {^[[:alpha:]]} $token]` (was `[string is 
> alpha $token]` - broke log10, atan2)
> here's the code for arrays:
>     # Function call or array reference?
>     set nexttok [lindex $::tokens $::tokpos]
>     if {$nexttok eq "(" && [regexp {^[[:alpha:]]} $token]} {
>         set fun [namespace which tcl::mathfunc::$token]
>         if {$fun ne {}} {
>             # It's a function
> incr ::tokpos
>             set opcodes "push $fun; "
>             append opcodes [parseFuncArgs]
>             return $opcodes
>         } else {
>             # Not a function, assume array reference
> incr ::tokpos
>             set opcodes "push $token; "
>             # Parse the index expression - leaves VALUE on stack
>             append opcodes [parse 0]
>             # Expect closing paren
>             set closing [lindex $::tokens $::tokpos]
>             if {$closing ne ")"} {
>                 error "Calc: expected ')' but found '$closing'"
>             }
>             # Stack now has: [arrayname, indexvalue]
> incr ::tokpos
>             append opcodes "loadArrayStk; "
>             return $opcodes
>         }
>     }
> In addition, there has indeed been some changes in the bytecode, land 
> and lor are no longer supported in 9.0 although they work in 8.6.
> I had an AI generate some 117 test cases, which all pass on 8.6 and 
> 111 on 9.x (the land/lor not being tested in 9.x).
> Colin, with your permission, I can post the code as a new file, with 
> all the test cases, say on a repository at github.
> I think a new TIP is worth considering; one that promotes assemble to 
> a supported form, with a compile and handle approach to avoid the time 
> parsing the ascii byte code text. I think that this would be great for 
> your = command, but also quite useful for others who might want to 
> create their own little languages.
> By doing it this way, it remains pure tcl, and avoids all the problems 
> with different systems and hardware that a binary extension would 
> create. In the end, I believe your code can achieve performance parity 
> with expr. Not only does it remove half the [expr {...}] baggage, but 
> all the $'s too! So much easier on these old eyes.
> Regards,
> Eric
>
> On Tue, Nov 4, 2025 at 1:06 PM EricT <tw...@gm...> wrote:
>
>     Hi Colin,
>
>     Hmmm, why can't you do bareword on $a(b) as a(b) you just need to
>     do an uplevel to see if a is a variable, if not, it would have to
>     be a function. True?
>
>     % tcl::unsupported::disassemble script {set a [expr {$b($c)}] }
>     snip
>
>     Command 2: "expr {$b($c)}..."
>         (2) push1 1 # "b"
>         (4) push1 2 # "c"
>         (6) loadStk
>         (7) loadArrayStk
>         (8) tryCvtToNumeric
>         (9) storeStk
>         (10) done
>
>     This doesn't look too much different from what you are producing.
>
>     I think what's really needed here is a TIP that would open up the
>     bytecode a bit so you don't need to use an unsupported command.
>     And then maybe even have a new command to take the string byte
>     code you are now producing and return a handle to a cached version
>     that was probably equivalent to the existing bytecode. Then your
>     cache array would be
>
>      set cache($exp) $handle
>
>     Instead of it having to parse the text, it could be as fast as
>     bytecode. You'd likely be just as fast as expr, and safe as well,
>     since you can't pass a string command in where the bareword is
>     required:
>
>      % set x {[pwd]}
>     [pwd]
>      % = sqrt(x)
>     exp= |sqrt(x)| code= |push ::tcl::mathfunc::sqrt; push x; loadStk;
>     invokeStk 2; | ifexist: 0
>     expected floating-point number but got "[pwd]"
>
>     I think you really have something here, perhaps this is the best
>     answer yet to slay the expr dragon!
>
>     Regards,
>
>     Eric
>
>     On Tue, Nov 4, 2025 at 6:52 AM Colin Macleod via Tcl-Core
>     <tcl...@li...> wrote:
>
>         Hi Eric,
>
>         That's very neat!
>
>         Yes, a pure Tcl version could go into TclLib. I still think it
>         may be worth trying a C implementation though.  The
>         work-around that's needed for array references [= 2* $a(b)]
>         would defeat the caching, so it would be good to speed up the
>         parsing if possible.  Also I think your caching may be
>         equivalent to doing byte-compilation, in which case it may
>         make sense to use the framework which already exists for that.
>
>         Colin.
>
>         On 04/11/2025 01:18, EricT wrote:
>
>             that is:
>
>             if {[info exist ::cache($exp)]} {
>
>                 tailcall ::tcl::unsupported::assemble $::cache($exp)
>             }
>
>             (hate gmail!)
>
>             On Mon, Nov 3, 2025 at 5:17 PM EricT <tw...@gm...> wrote:
>
>                 and silly of me, it should be:
>                 if {[info exist ::cache($exp)]} {
>                     tailcall ::tcl::unsupported::assemble $::cache($exp)
>                 }
>
>                 On Mon, Nov 3, 2025 at 4:50 PM EricT
>                 <tw...@gm...> wrote:
>
>                     With a debug line back in plus the tailcall:
>
>                     proc = args {
>
>                         set exp [join $args]
>                         if { [info exist ::cache($exp)] } {
>                              return [tailcall
>                      ::tcl::unsupported::assemble $::cache($exp)]
>                         }
>                         set tokens [tokenise $exp]
>                         deb1 "TOKENS = '$tokens'"
>                         set code [compile $tokens]
>                         deb1 "GENERATED CODE:\n$code\n"
>                         puts "exp= |$exp| code= |$code| ifexist: [info
>                     exist ::cache($exp)]"
>                         set ::cache($exp) $code
>                         uplevel [list ::tcl::unsupported::assemble $code]
>                     }
>
>                     % set a 5
>                     5
>                      % set b 10
>                     10
>                      % = a + b
>                     exp= |a + b| code= |push a; loadStk; push b;
>                     loadStk; add; | ifexist: 0
>                     15
>                     % = a + b
>                     15
>
>                     % time {= a + b} 1000
>                     1.73 microseconds per iteration
>
>                     Faster still!
>
>                     I thought the uplevel was needed to be able to get
>                     the local variables, seems not.
>
>                      % proc foo arg {set a 5; set b 10; set c [= a+b+arg]}
>                      % foo 5
>                     exp= |a+b+arg| code= |push a; loadStk; push b;
>                     loadStk; add; push arg; loadStk; add; | ifexist: 0
>                     20
>                     % foo 5
>                     20
>
>                      % proc foo arg {global xxx; set a 5; set b 10;
>                     set c [= a+b+arg+xxx]}
>
>                      % set xxx 100
>                     100
>                      % foo 200
>                     315
>                      % time {foo 200} 10000
>                     2.1775 microseconds per iteration
>
>                      % parray cache
>
>                     cache(a + b)       = push a; loadStk; push b;
>                     loadStk; add;
>                     cache(a+b+arg)     = push a; loadStk; push b;
>                     loadStk; add; push arg; loadStk; add;
>                     cache(a+b+arg+xxx) = push a; loadStk; push b;
>                     loadStk; add; push arg; loadStk; add; push xxx;
>                     loadStk; add;
>
>                     Very Impressive, great job Colin! Great catch Don!
>
>                     Eric
>
>                     On Mon, Nov 3, 2025 at 4:22 PM Donald Porter via
>                     Tcl-Core <tcl...@li...> wrote:
>
>                         Check what effect replacing [uplevel] with
>                         [tailcall] has.
>
>
>
>                             On Nov 3, 2025, at 7:13 PM, EricT
>                             <tw...@gm...> wrote:
>
>                             Subject: Your bytecode expression
>                             evaluator - impressive results with caching!
>
>                             Hey Colin:
>
>                             I took a look at your bytecode-based
>                             expression evaluator and was intrigued by
>                             the approach. I made a small modification
>                             to add caching and the results are really
>                             impressive. Here's what I changed:
>
>                             proc = args {
>                                 set exp [join $args]
>                                 if {[info exist ::cache($exp)]} {
>                                     return [uplevel [list
>                             ::tcl::unsupported::assemble $::cache($exp)]]
>                                 }
>                                 set tokens [tokenise $exp]
>                                 deb1 "TOKENS = '$tokens'"
>                                 set code [compile $tokens]
>                                 deb1 "GENERATED CODE:\n$code\n"
>                                 set ::cache($exp) $code
>                                 uplevel [list
>                             ::tcl::unsupported::assemble $code]
>                             }
>
>                             The cache is just a simple array lookup -
>                             one line to store, one line to retrieve.
>                             But the performance impact is huge:
>
>                             Performance Tests
>
>                             Without caching
>                             % time {= 1 + 2} 1000
>                             24.937 microseconds per iteration
>
>                             With caching
>                             % time {= 1 + 2} 1000
>                             1.8 microseconds per iteration
>
>                             That's a 13x speedup! The tokenize and
>                             parse steps were eating about 92% of the
>                             execution time.
>
>                             The Real Magic: Bare Variables + Caching
>
>                             What really impressed me is how well your
>                             bare variable feature synergizes with caching:
>
>                             % set a 5
>                             5
>                             % set b 6
>                             6
>                             % = a + b
>                             11
>                             % time {= a + b} 1000
>                             2.079 microseconds per iteration
>
>                             Now change the variable values
>                             % set a 10
>                             10
>                             % = a + b
>                             16
>                             % time {= a + b} 1000
>                             2.188 microseconds per iteration
>
>                             The cache entry stays valid even when the
>                             variable values change! Why? Because the
>                             bytecode stores variable names, not values:
>
>                             push a; loadStk; push b; loadStk; add;
>
>                             The loadStk instruction does runtime
>                             lookup, so:
>                             - Cache key is stable: "a + b"
>                             - Works for any values of a and b
>                             - One cache entry handles all value
>                             combinations
>
>                             Compare this to if we used $-substitution:
>
>                             = $a + $b    # With a=5, b=6 becomes "5 + 6"
>                             = $a + $b    # With a=10, b=6 becomes "10
>                             + 6" - different cache key!
>
>                             Every value change would create a new
>                             cache entry or worse, a cache miss.
>
>                             Comparison to Other Approaches
>
>                             Tcl's expr:  about 0.40 microseconds
>                             Direct C evaluator:  about 0.53 microseconds
>                             Your cached approach:  about 1.80 microseconds
>                             Your uncached approach:  about 24.9
>                             microseconds
>
>                             With caching, you're only 3-4x slower than
>                             a direct C evaluator.
>
>
>                             My Assessment
>
>                             Your design is excellent. The bare
>                             variable feature isn't just syntax sugar -
>                             it's essential for good cache performance.
>                             The synergy between:
>
>                             1. Bare variables leading to stable cache keys
>                             2. Runtime lookup keeping cache hot
>                             3. Simple caching providing dramatic speedup
>
>                             makes this really elegant.
>
>                             My recommendation: Keep it in Tcl! The
>                             implementation is clean, performance is
>                             excellent (1.8 microseconds is plenty
>                             fast), and converting to C would add
>                             significant complexity for minimal gain
>                             (maybe getting to about 1.0 microseconds).
>
>                             The Tcl prototype with caching is actually
>                             the right solution here. Sometimes the
>                             prototype IS the product!
>
>                             Excellent work on this. The bytecode
>                             approach really shines with caching enabled.
>
>                             On Sun, Nov 2, 2025 at 10:14 AM Colin
>                             Macleod via Tcl-Core
>                             <tcl...@li...> wrote:
>
>                                 Hi again,
>
>                                 I've now made a slightly more serious
>                                 prototype, see
>                                 https://cmacleod.me.uk/tcl/expr_ng
>
>                                 This is a modified version of the
>                                 prototype I wrote for tip 676. It's
>                                 still in Tcl, but doesn't use `expr`. 
>                                 It tokenises and parses the input,
>                                 then generates TAL bytecode and uses
>                                 ::tcl::unsupported::assemble to run
>                                 that.  A few examples:
>
>                                     (bin) 100 % set a [= 3.0/4]
>                                     0.75
>                                     (bin) 101 % set b [= sin(a*10)]
>                                     0.9379999767747389
>                                     (bin) 102 % set c [= (b-a)*100]
>                                     18.79999767747389
>                                     (bin) 103 % namespace eval nn {set
>                                     d [= 10**3]}
>                                     1000
>                                     (bin) 104 % set e [= a?nn::d:b]
>                                     1000
>                                     (bin) 105 % = {3 + [pwd]}
>                                     Calc: expected start of expression
>                                     but found '[pwd]'
>                                     (bin) 106 % = {3 + $q}
>                                     Calc: expected start of expression
>                                     but found '$q'
>                                     (bin) 107 % = sin (12)
>                                     -0.5365729180004349
>
>                                     (bin) 108 % array set rr {one 1
>                                     two 2 three 3}
>                                     (bin) 110 % = a * rr(two)
>                                     Calc: expected operator but found '('
>                                     (bin) 111 % = a * $rr(two)
>                                     1.5
>
>                                 - You can use $ to get an array value
>                                 substituted before the `=` code sees
>                                 the expression.
>
>                                     (bin) 112 % string repeat ! [=
>                                     nn::d / 15]
>                                     !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>
>                                 Colin.
>
>                                 On 02/11/2025 09:04, Donal Fellows wrote:
>
>                                     Doing the job properly would
>                                     definitely involve changing the
>                                     expression parser, with my
>                                     suggested fix being to turn all
>                                     bare words not otherwise
>                                     recognised as constants or in
>                                     positions that look like function
>                                     calls (it's a parser with some
>                                     lookahead) into simple variable
>                                     reads (NB: C resolves such
>                                     ambiguities within itself
>                                     differently, but that's one of the
>                                     nastiest parts of the language).
>                                     We would need to retain $ support
>                                     for resolving ambiguity (e.g.,
>                                     array reads vs function calls; you
>                                     can't safely inspect the
>                                     interpreter to resolve it at the
>                                     time of compiling the expression
>                                     due to traces and unknown
>                                     handlers) as well as
>                                     compatibility, but that's doable
>                                     as it is a change only in cases
>                                     that are currently errors.
>
>                                     Adding assignment is quite a bit
>                                     trickier, as that needs a new
>                                     major syntax class to describe the
>                                     left side of the assignment. I
>                                     suggest omitting that from
>                                     consideration at this stage.
>
>                                     Donal.
>
>                                     -------- Original message --------
>
>                                     From: Colin Macleod via Tcl-Core
>                                     <tcl...@li...>
>                                     <mailto:tcl...@li...>
>
>
>                                     Date: 02/11/2025 08:13 (GMT+00:00)
>
>                                     To: Pietro Cerutti <ga...@ga...>
>                                     <mailto:ga...@ga...>
>
>                                     Cc:
>                                     tcl...@li...,
>                                     av...@lo...
>
>                                     Subject: Re: [TCLCORE] Fwd: TIP
>                                     672 Implementation Complete -
>                                     Ready for Sponsorship
>
>                                     Indeed, this toy implementation
>                                     doesn't handle that:
>
>                                         % = sin (12)
>                                         can't read "sin": no such variable
>
>                                     I'm not sure that's serious, but
>                                     it could be fixed in a C
>                                     implementation.
>
>                                 _______________________________________________
>                                 Tcl-Core mailing list
>                                 Tcl...@li...
>                                 https://lists.sourceforge.net/lists/listinfo/tcl-core
>
>                             _______________________________________________
>                             Tcl-Core mailing list
>                             Tcl...@li...
>                             https://lists.sourceforge.net/lists/listinfo/tcl-core
>
>                         _______________________________________________
>                         Tcl-Core mailing list
>                         Tcl...@li...
>                         https://lists.sourceforge.net/lists/listinfo/tcl-core
>
>             _______________________________________________
>
>             Tcl-Core mailing list
>
>             Tcl...@li...
>
>             https://lists.sourceforge.net/lists/listinfo/tcl-core
>
>         _______________________________________________
>         Tcl-Core mailing list
>         Tcl...@li...
>         https://lists.sourceforge.net/lists/listinfo/tcl-core
>

Re: [TCLCORE] [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)

[Colin M. <col...@ya...>]

Hi Eric, thanks for your support.

For the boolean check I think it's more consistent to disallow 
alphabetic forms entirely and only accept numeric zero or non-zero, 
which is what boolean operations and functions return anyway. An 
alphabetic string should always be treated as a variable reference, no 
exceptions to worry about.

Similarly, i don't like the idea of  foo(bar)  sometimes being treated 
as a function call and sometimes as an array reference, depending on 
what definitions have been made elsewhere.  One should be able to tell 
what kind of thing it is just from the expression code, without 
searching for other definitions.  It is still possible to include an 
array reference by writing $foo(bar) so I would treat  foo(bar)  as a 
function call always, and fail if the function has not been defined.  
Sacrificing consistency for minor convenience is the slippery slope that 
leads to Perl.  :-)

You are welcome to post your modified version anywhere you like.

Personally I still want to try a C implementation, but that will take me 
a few weeks.

Best regards,
Colin.

On 05/11/2025 22:22, EricT wrote:
> Hi Colin,
>
> I've successfully modified your amazing code to handle arrays. In doing so, I also found 2 other issues, one is with your Boolean check, the other with your function name check, both because of [string is] issues.
>
> - Boolean check: `$tokeneq  "false" || $tokeneq  "true"` (was `[string isboolean  $token]` - treated 'f','n', 't', 'y',etc. variables asboolean  false, no, true, yes, ...)
>
> - Function check: `[regexp {^[[:alpha:]]} $token]` (was `[string is alpha $token]` - broke log10, atan2)
>
>
> here's the code for arrays:
>
>      # Function call or array reference?
>      setnexttok  [lindex  $::tokens $::tokpos]
>      if {$nexttok  eq  "(" && [regexp {^[[:alpha:]]} $token]} {
>          set fun [namespace  whichtcl::mathfunc::$token]
>          if {$fun ne {}} {
>              # It's a function
>              incr  ::tokpos
>              setopcodes  "push $fun; "
>              appendopcodes  [parseFuncArgs]
>              return $opcodes
>          } else {
>              # Not a function, assume array reference
>              incr  ::tokpos
>              setopcodes  "push $token; "
>              # Parse the index expression - leaves VALUE on stack
>              appendopcodes  [parse 0]
>              # Expect closingparen
>              set closing [lindex  $::tokens $::tokpos]
>              if {$closing ne ")"} {
>                  error "Calc: expected ')' but found '$closing'"
>              }
>              # Stack now has: [arrayname,indexvalue]
>              incr  ::tokpos      
>              appendopcodes  "loadArrayStk; "
>              return $opcodes
>          }
>      }
>
>
> In addition, there has indeed been some changes in thebytecode, land andlor  are no longer supported in 9.0 although they work in 8.6.
>
> I had an AI generate some 117 test cases, which all pass on 8.6 and 111 on 9.x (the land/lor  not being tested in 9.x).
>
> Colin, with your permission, I can post the code as a new file, with all the test cases, say on a repository atgithub.
>
> I think a new TIP is worth considering; one that promotes assemble to a supported form, with a compile and handle approach to avoid the time parsing theascii  byte code text. I think that this would be great for your = command, but also quite useful for others who might want to create their own little languages.
>
> By doing it this way, it remains puretcl, and avoids all the problems with different systems and hardware that a binary extension would create. In the end, I believe your code can achieve performance parity with expr. Not only does it remove half the [expr {...}] baggage, but all the $'s too! So much easier on these old eyes.
>
> Regards,
>
> Eric
>
>
> On Tue, Nov 4, 2025 at 1:06 PM EricT <tw...@gm...> wrote:
>
>     Hi Colin,
>
>     Hmmm, why can't you do bareword on $a(b) as a(b) you just need to
>     do an uplevel to see if a is a variable, if not, it would have to
>     be a function. True?
>
>     % tcl::unsupported::disassemble script {set a [expr {$b($c)}] }
>     snip
>       Command 2: "expr {$b($c)}..."
>         (2) push1 1 # "b"
>         (4) push1 2 # "c"
>         (6) loadStk
>         (7) loadArrayStk
>         (8) tryCvtToNumeric
>         (9) storeStk
>         (10) done
>
>     This doesn't look too much different from what you are producing.
>
>     I think what's really needed here is a TIP that would open up the
>     bytecode a bit so you don't need to use an unsupported command.
>     And then maybe even have a new command to take the string byte
>     code you are now producing and return a handle to a cached version
>     that was probably equivalent to the existing bytecode. Then your
>     cache array would be
>
>      set cache($exp) $handle
>
>     Instead of it having to parse the text, it could be as fast as
>     bytecode. You'd likely be just as fast as expr, and safe as well,
>     since you can't pass a string command in where the bareword is
>     required:
>
>      % set x {[pwd]}
>     [pwd]
>      % = sqrt(x)
>     exp= |sqrt(x)| code= |push ::tcl::mathfunc::sqrt; push x; loadStk;
>     invokeStk 2; | ifexist: 0
>     expected floating-point number but got "[pwd]"
>
>     I think you really have something here, perhaps this is the best
>     answer yet to slay the expr dragon!
>
>     Regards,
>
>     Eric
>
>
>     On Tue, Nov 4, 2025 at 6:52 AM Colin Macleod via Tcl-Core
>     <tcl...@li...> wrote:
>
>         Hi Eric,
>
>         That's very neat!
>
>         Yes, a pure Tcl version could go into TclLib. I still think it
>         may be worth trying a C implementation though.  The
>         work-around that's needed for array references [= 2* $a(b)]
>         would defeat the caching, so it would be good to speed up the
>         parsing if possible. Also I think your caching may be
>         equivalent to doing byte-compilation, in which case it may
>         make sense to use the framework which already exists for that.
>
>         Colin.
>
>         On 04/11/2025 01:18, EricT wrote:
>>         that is:
>>
>>         if {[info exist ::cache($exp)]} {
>>             tailcall ::tcl::unsupported::assemble $::cache($exp)
>>         }
>>
>>         (hate gmail!)
>>
>>
>>         On Mon, Nov 3, 2025 at 5:17 PM EricT <tw...@gm...> wrote:
>>
>>             and silly of me, it should be:
>>             if {[info exist ::cache($exp)]} {
>>                 tailcall ::tcl::unsupported::assemble $::cache($exp)
>>             }
>>
>>
>>             On Mon, Nov 3, 2025 at 4:50 PM EricT <tw...@gm...>
>>             wrote:
>>
>>                 With a debug line back in plus the tailcall:
>>
>>                 proc = args {
>>                     set exp [join $args]
>>                     if { [info exist ::cache($exp)] } {
>>                          return [tailcall
>>                  ::tcl::unsupported::assemble $::cache($exp)]
>>                     }
>>                     set tokens [tokenise $exp]
>>                     deb1 "TOKENS = '$tokens'"
>>                     set code [compile $tokens]
>>                     deb1 "GENERATED CODE:\n$code\n"
>>                     puts "exp= |$exp| code= |$code| ifexist: [info
>>                 exist ::cache($exp)]"
>>                     set ::cache($exp) $code
>>                     uplevel [list ::tcl::unsupported::assemble $code]
>>                 }
>>
>>                 % set a 5
>>                 5
>>                  % set b 10
>>                 10
>>                  % = a + b
>>                 exp= |a + b| code= |push a; loadStk; push b; loadStk;
>>                 add; | ifexist: 0
>>                 15
>>                 % = a + b
>>                 15
>>
>>                 % time {= a + b} 1000
>>                 1.73 microseconds per iteration
>>
>>
>>                 Faster still!
>>
>>                 I thought the uplevel was needed to be able to get
>>                 the local variables, seems not.
>>
>>                  % proc foo arg {set a 5; set b 10; set c [= a+b+arg]}
>>                  % foo 5
>>                 exp= |a+b+arg| code= |push a; loadStk; push b;
>>                 loadStk; add; push arg; loadStk; add; | ifexist: 0
>>                 20
>>                 % foo 5
>>                 20
>>
>>                  % proc foo arg {global xxx; set a 5; set b 10; set c
>>                 [= a+b+arg+xxx]}
>>
>>                  % set xxx 100
>>                 100
>>                  % foo 200
>>                 315
>>                  % time {foo 200} 10000
>>                 2.1775 microseconds per iteration
>>
>>                  % parray cache
>>                 cache(a + b)       = push a; loadStk; push b;
>>                 loadStk; add;
>>                 cache(a+b+arg)     = push a; loadStk; push b;
>>                 loadStk; add; push arg; loadStk; add;
>>                 cache(a+b+arg+xxx) = push a; loadStk; push b;
>>                 loadStk; add; push arg; loadStk; add; push xxx;
>>                 loadStk; add;
>>
>>
>>                 Very Impressive, great job Colin! Great catch Don!
>>
>>                 Eric
>>
>>
>>
>>
>>                 On Mon, Nov 3, 2025 at 4:22 PM Donald Porter via
>>                 Tcl-Core <tcl...@li...> wrote:
>>
>>                     Check what effect replacing [uplevel] with
>>                     [tailcall] has.
>>
>>>                     On Nov 3, 2025, at 7:13 PM, EricT
>>>                     <tw...@gm...> wrote:
>>>
>>>                     Subject: Your bytecode expression evaluator -
>>>                     impressive results with caching!
>>>
>>>                     Hey Colin:
>>>
>>>                     I took a look at your bytecode-based expression
>>>                     evaluator and was intrigued by the approach. I
>>>                     made a small modification to add caching and the
>>>                     results are really impressive. Here's what I
>>>                     changed:
>>>
>>>                     proc = args {
>>>                         set exp [join $args]
>>>                         if {[info exist ::cache($exp)]} {
>>>                             return [uplevel [list
>>>                     ::tcl::unsupported::assemble $::cache($exp)]]
>>>                         }
>>>                         set tokens [tokenise $exp]
>>>                         deb1 "TOKENS = '$tokens'"
>>>                         set code [compile $tokens]
>>>                         deb1 "GENERATED CODE:\n$code\n"
>>>                         set ::cache($exp) $code
>>>                         uplevel [list ::tcl::unsupported::assemble
>>>                     $code]
>>>                     }
>>>
>>>                     The cache is just a simple array lookup - one
>>>                     line to store, one line to retrieve. But the
>>>                     performance impact is huge:
>>>
>>>                     Performance Tests
>>>
>>>                     Without caching
>>>                     % time {= 1 + 2} 1000
>>>                     24.937 microseconds per iteration
>>>
>>>                     With caching
>>>                     % time {= 1 + 2} 1000
>>>                     1.8 microseconds per iteration
>>>
>>>                     That's a 13x speedup! The tokenize and parse
>>>                     steps were eating about 92% of the execution time.
>>>
>>>                     The Real Magic: Bare Variables + Caching
>>>
>>>                     What really impressed me is how well your bare
>>>                     variable feature synergizes with caching:
>>>
>>>                     % set a 5
>>>                     5
>>>                     % set b 6
>>>                     6
>>>                     % = a + b
>>>                     11
>>>                     % time {= a + b} 1000
>>>                     2.079 microseconds per iteration
>>>
>>>                     Now change the variable values
>>>                     % set a 10
>>>                     10
>>>                     % = a + b
>>>                     16
>>>                     % time {= a + b} 1000
>>>                     2.188 microseconds per iteration
>>>
>>>                     The cache entry stays valid even when the
>>>                     variable values change! Why? Because the
>>>                     bytecode stores variable names, not values:
>>>
>>>                     push a; loadStk; push b; loadStk; add;
>>>
>>>                     The loadStk instruction does runtime lookup, so:
>>>                     - Cache key is stable: "a + b"
>>>                     - Works for any values of a and b
>>>                     - One cache entry handles all value combinations
>>>
>>>                     Compare this to if we used $-substitution:
>>>
>>>                     = $a + $b    # With a=5, b=6 becomes "5 + 6"
>>>                     = $a + $b    # With a=10, b=6 becomes "10 + 6" -
>>>                     different cache key!
>>>
>>>                     Every value change would create a new cache
>>>                     entry or worse, a cache miss.
>>>
>>>                     Comparison to Other Approaches
>>>
>>>                     Tcl's expr:              about 0.40 microseconds
>>>                     Direct C evaluator:      about 0.53 microseconds
>>>                     Your cached approach:    about 1.80 microseconds
>>>                     Your uncached approach:  about 24.9 microseconds
>>>
>>>                     With caching, you're only 3-4x slower than a
>>>                     direct C evaluator.
>>>
>>>
>>>                     My Assessment
>>>
>>>                     Your design is excellent. The bare variable
>>>                     feature isn't just syntax sugar - it's essential
>>>                     for good cache performance. The synergy between:
>>>
>>>                     1. Bare variables leading to stable cache keys
>>>                     2. Runtime lookup keeping cache hot
>>>                     3. Simple caching providing dramatic speedup
>>>
>>>                     makes this really elegant.
>>>
>>>                     My recommendation: Keep it in Tcl! The
>>>                     implementation is clean, performance is
>>>                     excellent (1.8 microseconds is plenty fast), and
>>>                     converting to C would add significant complexity
>>>                     for minimal gain (maybe getting to about 1.0
>>>                     microseconds).
>>>
>>>                     The Tcl prototype with caching is actually the
>>>                     right solution here. Sometimes the prototype IS
>>>                     the product!
>>>
>>>                     Excellent work on this. The bytecode approach
>>>                     really shines with caching enabled.
>>>
>>>                     On Sun, Nov 2, 2025 at 10:14 AM Colin Macleod
>>>                     via Tcl-Core <tcl...@li...> wrote:
>>>
>>>                         Hi again,
>>>
>>>                         I've now made a slightly more serious
>>>                         prototype, see
>>>                         https://cmacleod.me.uk/tcl/expr_ng
>>>
>>>                         This is a modified version of the prototype
>>>                         I wrote for tip 676. It's still in Tcl, but
>>>                         doesn't use `expr`.  It tokenises and parses
>>>                         the input, then generates TAL bytecode and
>>>                         uses ::tcl::unsupported::assemble to run
>>>                         that.  A few examples:
>>>
>>>                             (bin) 100 % set a [= 3.0/4]
>>>                             0.75
>>>                             (bin) 101 % set b [= sin(a*10)]
>>>                             0.9379999767747389
>>>                             (bin) 102 % set c [= (b-a)*100]
>>>                             18.79999767747389
>>>                             (bin) 103 % namespace eval nn {set d [=
>>>                             10**3]}
>>>                             1000
>>>                             (bin) 104 % set e [= a?nn::d:b]
>>>                             1000
>>>                             (bin) 105 % = {3 + [pwd]}
>>>                             Calc: expected start of expression but
>>>                             found '[pwd]'
>>>                             (bin) 106 % = {3 + $q}
>>>                             Calc: expected start of expression but
>>>                             found '$q'
>>>                             (bin) 107 % = sin (12)
>>>                             -0.5365729180004349
>>>
>>>                             (bin) 108 % array set rr {one 1 two 2
>>>                             three 3}
>>>                             (bin) 110 % = a * rr(two)
>>>                             Calc: expected operator but found '('
>>>                             (bin) 111 % = a * $rr(two)
>>>                             1.5
>>>
>>>                         - You can use $ to get an array value
>>>                         substituted before the `=` code sees the
>>>                         expression.
>>>
>>>                             (bin) 112 % string repeat ! [= nn::d / 15]
>>>                             !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>>>
>>>                         Colin.
>>>
>>>                         On 02/11/2025 09:04, Donal Fellows wrote:
>>>>                         Doing the job properly would definitely
>>>>                         involve changing the expression parser,
>>>>                         with my suggested fix being to turn all
>>>>                         bare words not otherwise recognised as
>>>>                         constants or in positions that look like
>>>>                         function calls (it's a parser with some
>>>>                         lookahead) into simple variable reads (NB:
>>>>                         C resolves such ambiguities within itself
>>>>                         differently, but that's one of the nastiest
>>>>                         parts of the language). We would need to
>>>>                         retain $ support for resolving ambiguity
>>>>                         (e.g., array reads vs function calls; you
>>>>                         can't safely inspect the interpreter to
>>>>                         resolve it at the time of compiling the
>>>>                         expression due to traces and unknown
>>>>                         handlers) as well as compatibility, but
>>>>                         that's doable as it is a change only in
>>>>                         cases that are currently errors.
>>>>
>>>>                         Adding assignment is quite a bit trickier,
>>>>                         as that needs a new major syntax class to
>>>>                         describe the left side of the assignment. I
>>>>                         suggest omitting that from consideration at
>>>>                         this stage.
>>>>
>>>>                         Donal.
>>>>
>>>>                         -------- Original message --------
>>>>                         From: Colin Macleod via Tcl-Core
>>>>                         <tcl...@li...>
>>>>                         <mailto:tcl...@li...>
>>>>                         Date: 02/11/2025 08:13 (GMT+00:00)
>>>>                         To: Pietro Cerutti <ga...@ga...>
>>>>                         <mailto:ga...@ga...>
>>>>                         Cc: tcl...@li...,
>>>>                         av...@lo...
>>>>                         Subject: Re: [TCLCORE] Fwd: TIP 672
>>>>                         Implementation Complete - Ready for
>>>>                         Sponsorship
>>>>
>>>>                         Indeed, this toy implementation doesn't
>>>>                         handle that:
>>>>
>>>>                             % = sin (12)
>>>>                             can't read "sin": no such variable
>>>>
>>>>                         I'm not sure that's serious, but it could
>>>>                         be fixed in a C implementation.
>>>>
>>>                         _______________________________________________
>>>                         Tcl-Core mailing list
>>>                         Tcl...@li...
>>>                         https://lists.sourceforge.net/lists/listinfo/tcl-core
>>>
>>>                     _______________________________________________
>>>                     Tcl-Core mailing list
>>>                     Tcl...@li...
>>>                     https://lists.sourceforge.net/lists/listinfo/tcl-core
>>
>>                     _______________________________________________
>>                     Tcl-Core mailing list
>>                     Tcl...@li...
>>                     https://lists.sourceforge.net/lists/listinfo/tcl-core
>>
>>
>>
>>         _______________________________________________
>>         Tcl-Core mailing list
>>         Tcl...@li...
>>         https://lists.sourceforge.net/lists/listinfo/tcl-core
>         _______________________________________________
>         Tcl-Core mailing list
>         Tcl...@li...
>         https://lists.sourceforge.net/lists/listinfo/tcl-core
>

Re: [TCLCORE] Tcl/Tk 9.0.3 Release Candidate

[<apn...@ya...>]

Win10 + VS 2022, x64 and x86
make test and install+load
No unexpected failures in either Tcl, Tk or packages (except db driver
unavailability)

-----Original Message-----
From: Jan Nijtmans <jan...@gm...> 
Sent: Wednesday, November 5, 2025 6:47 PM
To: Tcl Core List <tcl...@li...>
Subject: [TCLCORE] Tcl/Tk 9.0.3 Release Candidate

Now available at

https://sourceforge.net/projects/tcl/files/Tcl/9.0.3/

are RC0 candidate source code distribution pre-releases of Tcl/Tk 9.0.3

This is the first candidate release leading to the release of Tcl/Tk
9.0.3. Testing of builds
and operations on multiple platforms is invited.  Any critical problem
that should block
the release should be reported immediately.

Only the full *-src.tar.gz files are there, the zip-files and the
stripped-down
versions will follow.

Preliminary release notes are available as well. Please report any
suggestions/improvements you may find.

Thank you for your contributions and assistance.
           Jan Nijtmans


_______________________________________________
Tcl-Core mailing list
Tcl...@li...
https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] Tcl/Tk 9.0.3 Release Candidate

[Jan N. <jan...@gm...>]

Op do 6 nov 2025 om 06:45 schreef Christian Werner
<Chr...@t-...>:

> Both ways are contradictory to the objective, which is to support multiple
> displays plus weird combinations of visuals and colormaps. So what about
> just using the much-loved TCL_UNUSED macro there? Or even strip off that
> parameter in InitCacheWindow()?

That was the advice I needed. Thanks! So it looks like cache->tkwin
is supposed to be the main Tk_Window of the application, while
tkwin is the current one.

TCL_UNUSED should only be used when we cannot simply eliminate
the parameter. Nicht im frage hier.

Again, thanks!
          Jan Nijtmans

Re: [TCLCORE] [Ext] Re: [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)

[Zaumseil R. <RZa...@kk...>]

Hello

I also like the = approach as a new expr command without need of $ for variables.
But I'm not sure about the currently proposed syntax.
Is it "= 1 + 2" or "= 1+2" (with or without spaces)?
If it is the later then I would propose to use some of the syntax of the "let" from tip 674.

Some syntax ideas:
"= 1+2" return 3, simple case
"= a 1+2 b 2+4 c a+b" return {3 6 9} and set a,b,c
"= 1+2 = 2+4" return {3 6}
And may be some others…

Regards
rene
Von: EricT <tw...@gm...>
Gesendet: Mittwoch, 5. November 2025 23:22
An: Colin Macleod <col...@ya...>; tcl...@li...
Betreff: [Ext] Re: [TCLCORE] [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)


Hi Colin,



I've successfully modified your amazing code to handle arrays. In doing so, I also found 2 other issues, one is with your Boolean check, the other with your function name check, both because of [string is] issues.



- Boolean check: `$token eq "false" || $token eq "true"` (was `[string is boolean $token]` - treated 'f','n', 't', 'y', etc. variables as boolean false, no, true, yes, ...)



- Function check: `[regexp {^[[:alpha:]]} $token]` (was `[string is alpha $token]` - broke log10, atan2)





here's the code for arrays:



    # Function call or array reference?

    set nexttok [lindex $::tokens $::tokpos]

    if {$nexttok eq "(" && [regexp {^[[:alpha:]]} $token]} {

        set fun [namespace which tcl::mathfunc::$token]

        if {$fun ne {}} {

            # It's a function

            incr ::tokpos

            set opcodes "push $fun; "

            append opcodes [parseFuncArgs]

            return $opcodes

        } else {

            # Not a function, assume array reference

            incr ::tokpos

            set opcodes "push $token; "

            # Parse the index expression - leaves VALUE on stack

            append opcodes [parse 0]

            # Expect closing paren

            set closing [lindex $::tokens $::tokpos]

            if {$closing ne ")"} {

                error "Calc: expected ')' but found '$closing'"

            }

            # Stack now has: [arrayname, indexvalue]

            incr ::tokpos

            append opcodes "loadArrayStk; "

            return $opcodes

        }

    }





In addition, there has indeed been some changes in the bytecode, land and lor are no longer supported in 9.0 although they work in 8.6.



I had an AI generate some 117 test cases, which all pass on 8.6 and 111 on 9.x (the land/lor not being tested in 9.x).



Colin, with your permission, I can post the code as a new file, with all the test cases, say on a repository at github.



I think a new TIP is worth considering; one that promotes assemble to a supported form, with a compile and handle approach to avoid the time parsing the ascii byte code text. I think that this would be great for your = command, but also quite useful for others who might want to create their own little languages.



By doing it this way, it remains pure tcl, and avoids all the problems with different systems and hardware that a binary extension would create. In the end, I believe your code can achieve performance parity with expr. Not only does it remove half the [expr {...}] baggage, but all the $'s too! So much easier on these old eyes.



Regards,



Eric


On Tue, Nov 4, 2025 at 1:06 PM EricT <tw...@gm...<mailto:tw...@gm...>> wrote:
Hi Colin,

Hmmm, why can't you do bareword on $a(b) as a(b) you just need to do an uplevel to see if a is a variable, if not, it would have to be a function. True?

% tcl::unsupported::disassemble script {set a [expr {$b($c)}] }
snip
  Command 2: "expr {$b($c)}..."
    (2) push1 1 # "b"
    (4) push1 2 # "c"
    (6) loadStk
    (7) loadArrayStk
    (8) tryCvtToNumeric
    (9) storeStk
    (10) done
This doesn't look too much different from what you are producing.

I think what's really needed here is a TIP that would open up the bytecode a bit so you don't need to use an unsupported command. And then maybe even have a new command to take the string byte code you are now producing and return a handle to a cached version that was probably equivalent to the existing bytecode. Then your cache array would be

 set cache($exp) $handle

Instead of it having to parse the text, it could be as fast as bytecode. You'd likely be just as fast as expr, and safe as well, since you can't pass a string command in where the bareword is required:

 % set x {[pwd]}
[pwd]
 % = sqrt(x)
exp= |sqrt(x)| code= |push ::tcl::mathfunc::sqrt; push x; loadStk; invokeStk 2; | ifexist: 0
expected floating-point number but got "[pwd]"
I think you really have something here, perhaps this is the best answer yet to slay the expr dragon!

Regards,

Eric


On Tue, Nov 4, 2025 at 6:52 AM Colin Macleod via Tcl-Core <tcl...@li...<mailto:tcl...@li...>> wrote:

Hi Eric,

That's very neat!

Yes, a pure Tcl version could go into TclLib. I still think it may be worth trying a C implementation though.  The work-around that's needed for array references [= 2* $a(b)] would defeat the caching, so it would be good to speed up the parsing if possible.  Also I think your caching may be equivalent to doing byte-compilation, in which case it may make sense to use the framework which already exists for that.

Colin.
On 04/11/2025 01:18, EricT wrote:
that is:

if {[info exist ::cache($exp)]} {
    tailcall ::tcl::unsupported::assemble $::cache($exp)
}

(hate gmail!)


On Mon, Nov 3, 2025 at 5:17 PM EricT <tw...@gm...<mailto:tw...@gm...>> wrote:
and silly of me, it should be:
if {[info exist ::cache($exp)]} {
    tailcall ::tcl::unsupported::assemble $::cache($exp)
}

On Mon, Nov 3, 2025 at 4:50 PM EricT <tw...@gm...<mailto:tw...@gm...>> wrote:
With a debug line back in plus the tailcall:

proc = args {
    set exp [join $args]
    if { [info exist ::cache($exp)] } {
         return [tailcall  ::tcl::unsupported::assemble $::cache($exp)]
    }
    set tokens [tokenise $exp]
    deb1 "TOKENS = '$tokens'"
    set code [compile $tokens]
    deb1 "GENERATED CODE:\n$code\n"
    puts "exp= |$exp| code= |$code| ifexist: [info exist ::cache($exp)]"
    set ::cache($exp) $code
    uplevel [list ::tcl::unsupported::assemble $code]
}

% set a 5
5
 % set b 10
10
 % = a + b
exp= |a + b| code= |push a; loadStk; push b; loadStk; add; | ifexist: 0
15
% = a + b
15

% time {= a + b} 1000
1.73 microseconds per iteration

Faster still!

I thought the uplevel was needed to be able to get the local variables, seems not.

 % proc foo arg {set a 5; set b 10; set c [= a+b+arg]}
 % foo 5
exp= |a+b+arg| code= |push a; loadStk; push b; loadStk; add; push arg; loadStk; add; | ifexist: 0
20
% foo 5
20

 % proc foo arg {global xxx; set a 5; set b 10; set c [= a+b+arg+xxx]}

 % set xxx 100
100
 % foo 200
315
 % time {foo 200} 10000
2.1775 microseconds per iteration

 % parray cache
cache(a + b)       = push a; loadStk; push b; loadStk; add;
cache(a+b+arg)     = push a; loadStk; push b; loadStk; add; push arg; loadStk; add;
cache(a+b+arg+xxx) = push a; loadStk; push b; loadStk; add; push arg; loadStk; add; push xxx; loadStk; add;

Very Impressive, great job Colin! Great catch Don!

Eric




On Mon, Nov 3, 2025 at 4:22 PM Donald Porter via Tcl-Core <tcl...@li...<mailto:tcl...@li...>> wrote:
Check what effect replacing [uplevel] with [tailcall] has.


On Nov 3, 2025, at 7:13 PM, EricT <tw...@gm...<mailto:tw...@gm...>> wrote:

Subject: Your bytecode expression evaluator - impressive results with caching!

Hey Colin:

I took a look at your bytecode-based expression evaluator and was intrigued by the approach. I made a small modification to add caching and the results are really impressive. Here's what I changed:

proc = args {
    set exp [join $args]
    if {[info exist ::cache($exp)]} {
        return [uplevel [list ::tcl::unsupported::assemble $::cache($exp)]]
    }
    set tokens [tokenise $exp]
    deb1 "TOKENS = '$tokens'"
    set code [compile $tokens]
    deb1 "GENERATED CODE:\n$code\n"
    set ::cache($exp) $code
    uplevel [list ::tcl::unsupported::assemble $code]
}

The cache is just a simple array lookup - one line to store, one line to retrieve. But the performance impact is huge:

Performance Tests

Without caching
% time {= 1 + 2} 1000
24.937 microseconds per iteration

With caching
% time {= 1 + 2} 1000
1.8 microseconds per iteration

That's a 13x speedup! The tokenize and parse steps were eating about 92% of the execution time.

The Real Magic: Bare Variables + Caching

What really impressed me is how well your bare variable feature synergizes with caching:

% set a 5
5
% set b 6
6
% = a + b
11
% time {= a + b} 1000
2.079 microseconds per iteration

Now change the variable values
% set a 10
10
% = a + b
16
% time {= a + b} 1000
2.188 microseconds per iteration

The cache entry stays valid even when the variable values change! Why? Because the bytecode stores variable names, not values:

push a; loadStk; push b; loadStk; add;

The loadStk instruction does runtime lookup, so:
- Cache key is stable: "a + b"
- Works for any values of a and b
- One cache entry handles all value combinations

Compare this to if we used $-substitution:

= $a + $b    # With a=5, b=6 becomes "5 + 6"
= $a + $b    # With a=10, b=6 becomes "10 + 6" - different cache key!

Every value change would create a new cache entry or worse, a cache miss.

Comparison to Other Approaches

Tcl's expr:              about 0.40 microseconds
Direct C evaluator:      about 0.53 microseconds
Your cached approach:    about 1.80 microseconds
Your uncached approach:  about 24.9 microseconds

With caching, you're only 3-4x slower than a direct C evaluator.


My Assessment

Your design is excellent. The bare variable feature isn't just syntax sugar - it's essential for good cache performance. The synergy between:

1. Bare variables leading to stable cache keys
2. Runtime lookup keeping cache hot
3. Simple caching providing dramatic speedup

makes this really elegant.

My recommendation: Keep it in Tcl! The implementation is clean, performance is excellent (1.8 microseconds is plenty fast), and converting to C would add significant complexity for minimal gain (maybe getting to about 1.0 microseconds).

The Tcl prototype with caching is actually the right solution here. Sometimes the prototype IS the product!

Excellent work on this. The bytecode approach really shines with caching enabled.

On Sun, Nov 2, 2025 at 10:14 AM Colin Macleod via Tcl-Core <tcl...@li...<mailto:tcl...@li...>> wrote:

Hi again,

I've now made a slightly more serious prototype, see https://cmacleod.me.uk/tcl/expr_ng

This is a modified version of the prototype I wrote for tip 676. It's still in Tcl, but doesn't use `expr`.  It tokenises and parses the input, then generates TAL bytecode and uses ::tcl::unsupported::assemble to run that.  A few examples:

(bin) 100 % set a [= 3.0/4]
0.75
(bin) 101 % set b [= sin(a*10)]
0.9379999767747389
(bin) 102 % set c [= (b-a)*100]
18.79999767747389
(bin) 103 % namespace eval nn {set d [= 10**3]}
1000
(bin) 104 % set e [= a?nn::d:b]
1000
(bin) 105 % = {3 + [pwd]}
Calc: expected start of expression but found '[pwd]'
(bin) 106 % = {3 + $q}
Calc: expected start of expression but found '$q'
(bin) 107 % = sin (12)
-0.5365729180004349

(bin) 108 % array set rr {one 1 two 2 three 3}
(bin) 110 % = a * rr(two)
Calc: expected operator but found '('
(bin) 111 % = a * $rr(two)
1.5

- You can use $ to get an array value substituted before the `=` code sees the expression.

(bin) 112 % string repeat ! [= nn::d / 15]
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Colin.
On 02/11/2025 09:04, Donal Fellows wrote:
Doing the job properly would definitely involve changing the expression parser, with my suggested fix being to turn all bare words not otherwise recognised as constants or in positions that look like function calls (it's a parser with some lookahead) into simple variable reads (NB: C resolves such ambiguities within itself differently, but that's one of the nastiest parts of the language). We would need to retain $ support for resolving ambiguity (e.g., array reads vs function calls; you can't safely inspect the interpreter to resolve it at the time of compiling the expression due to traces and unknown handlers) as well as compatibility, but that's doable as it is a change only in cases that are currently errors.

Adding assignment is quite a bit trickier, as that needs a new major syntax class to describe the left side of the assignment. I suggest omitting that from consideration at this stage.

Donal.

-------- Original message --------
From: Colin Macleod via Tcl-Core <tcl...@li...><mailto:tcl...@li...>
Date: 02/11/2025 08:13 (GMT+00:00)
To: Pietro Cerutti <ga...@ga...><mailto:ga...@ga...>
Cc: tcl...@li...<mailto:tcl...@li...>, av...@lo...<mailto:av...@lo...>
Subject: Re: [TCLCORE] Fwd: TIP 672 Implementation Complete - Ready for Sponsorship


Indeed, this toy implementation doesn't handle that:

% = sin (12)
can't read "sin": no such variable

I'm not sure that's serious, but it could be fixed in a C implementation.
_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core
_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core

_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core


_______________________________________________

Tcl-Core mailing list

Tcl...@li...<mailto:Tcl...@li...>

https://lists.sourceforge.net/lists/listinfo/tcl-core
_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] Tcl/Tk 9.0.3 Release Candidate

[Christian W. <Chr...@t-...>]

On 11/06/2025 12:18 AM, Jan Nijtmans wrote:
> Op wo 5 nov 2025 om 21:06 schreef Paul Obermeier <pa...@po...>:
>> 1 warning during Tk compilation:
>>      Tk/generic/ttk/ttkCache.c: In function 'InitCacheWindow':
>>      Tk/generic/ttk/ttkCache.c:256:64: warning: unused parameter 'tkwin' [-Wunused-parameter]
>>      static void InitCacheWindow(Ttk_ResourceCache cache, Tk_Window tkwin)
>
> There are two ways to solve this:
> 1) change line 259 from
>         cache->tkwin = Tk_MainWindow(cache->interp);
>     to
>         cache->tkwin = tkwin;
>     this was how the line looked like originally.
>
> 2)  Replace separate 'tkwin' arguments with 'cache->tkwin' in more places.
>      Experiment:
>             https://core.tcl-lang.org/tk/info/ed67b8e0980fd5af
>
> Does anyone (Christian???) have any advice here?

Both ways are contradictory to the objective, which is to support multiple
displays plus weird combinations of visuals and colormaps. So what about
just using the much-loved TCL_UNUSED macro there? Or even strip off that
parameter in InitCacheWindow()?

Regards,
Christian

Re: [TCLCORE] Tcl/Tk 9.0.3 Release Candidate

[Jan N. <jan...@gm...>]

Op wo 5 nov 2025 om 21:06 schreef Paul Obermeier <pa...@po...>:
> 1 warning during Tk compilation:
>      Tk/generic/ttk/ttkCache.c: In function 'InitCacheWindow':
>      Tk/generic/ttk/ttkCache.c:256:64: warning: unused parameter 'tkwin' [-Wunused-parameter]
>      static void InitCacheWindow(Ttk_ResourceCache cache, Tk_Window tkwin)

There are two ways to solve this:
1) change line 259 from
        cache->tkwin = Tk_MainWindow(cache->interp);
    to
        cache->tkwin = tkwin;
    this was how the line looked like originally.

2)  Replace separate 'tkwin' arguments with 'cache->tkwin' in more places.
     Experiment:
            https://core.tcl-lang.org/tk/info/ed67b8e0980fd5af

Does anyone (Christian???) have any advice here?

Regards,
     Jan Nijtmans

Re: [TCLCORE] [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)

[EricT <tw...@gm...>]

Hi Colin,

I've successfully modified your amazing code to handle arrays. In
doing so, I also found 2 other issues, one is with your Boolean check,
the other with your function name check, both because of [string is]
issues.

- Boolean check: `$token eq "false" || $token eq "true"` (was `[string
is boolean $token]` - treated 'f','n', 't', 'y', etc. variables as
boolean false, no, true, yes, ...)

- Function check: `[regexp {^[[:alpha:]]} $token]` (was `[string is
alpha $token]` - broke log10, atan2)


here's the code for arrays:

    # Function call or array reference?
    set nexttok [lindex $::tokens $::tokpos]
    if {$nexttok eq "(" && [regexp {^[[:alpha:]]} $token]} {
        set fun [namespace which tcl::mathfunc::$token]
        if {$fun ne {}} {
            # It's a function
            incr ::tokpos
            set opcodes "push $fun; "
            append opcodes [parseFuncArgs]
            return $opcodes
        } else {
            # Not a function, assume array reference
            incr ::tokpos
            set opcodes "push $token; "
            # Parse the index expression - leaves VALUE on stack
            append opcodes [parse 0]
            # Expect closing paren
            set closing [lindex $::tokens $::tokpos]
            if {$closing ne ")"} {
                error "Calc: expected ')' but found '$closing'"
            }
            # Stack now has: [arrayname, indexvalue]
            incr ::tokpos
            append opcodes "loadArrayStk; "
            return $opcodes
        }
    }


In addition, there has indeed been some changes in the bytecode, land
and lor are no longer supported in 9.0 although they work in 8.6.

I had an AI generate some 117 test cases, which all pass on 8.6 and
111 on 9.x (the land/lor not being tested in 9.x).

Colin, with your permission, I can post the code as a new file, with
all the test cases, say on a repository at github.

I think a new TIP is worth considering; one that promotes assemble to
a supported form, with a compile and handle approach to avoid the time
parsing the ascii byte code text. I think that this would be great for
your = command, but also quite useful for others who might want to
create their own little languages.

By doing it this way, it remains pure tcl, and avoids all the problems
with different systems and hardware that a binary extension would
create. In the end, I believe your code can achieve performance parity
with expr. Not only does it remove half the [expr {...}] baggage, but
all the $'s too! So much easier on these old eyes.

Regards,

Eric



On Tue, Nov 4, 2025 at 1:06 PM EricT <tw...@gm...> wrote:

> Hi Colin,
>
> Hmmm, why can't you do bareword on $a(b) as a(b) you just need to do an
> uplevel to see if a is a variable, if not, it would have to be a function.
> True?
>
> % tcl::unsupported::disassemble script {set a [expr {$b($c)}] }
> snip
>   Command 2: "expr {$b($c)}..."
>     (2) push1 1 # "b"
>     (4) push1 2 # "c"
>     (6) loadStk
>     (7) loadArrayStk
>     (8) tryCvtToNumeric
>     (9) storeStk
>     (10) done
>
> This doesn't look too much different from what you are producing.
>
> I think what's really needed here is a TIP that would open up the bytecode
> a bit so you don't need to use an unsupported command. And then maybe even
> have a new command to take the string byte code you are now producing and
> return a handle to a cached version that was probably equivalent to the
> existing bytecode. Then your cache array would be
>
>  set cache($exp) $handle
>
> Instead of it having to parse the text, it could be as fast as bytecode.
> You'd likely be just as fast as expr, and safe as well, since you can't
> pass a string command in where the bareword is required:
>
>  % set x {[pwd]}
> [pwd]
>  % = sqrt(x)
> exp= |sqrt(x)| code= |push ::tcl::mathfunc::sqrt; push x; loadStk;
> invokeStk 2; | ifexist: 0
> expected floating-point number but got "[pwd]"
>
> I think you really have something here, perhaps this is the best answer
> yet to slay the expr dragon!
>
> Regards,
>
> Eric
>
>
> On Tue, Nov 4, 2025 at 6:52 AM Colin Macleod via Tcl-Core <
> tcl...@li...> wrote:
>
>> Hi Eric,
>>
>> That's very neat!
>>
>> Yes, a pure Tcl version could go into TclLib. I still think it may be
>> worth trying a C implementation though.  The work-around that's needed for
>> array references [= 2* $a(b)] would defeat the caching, so it would be good
>> to speed up the parsing if possible.  Also I think your caching may be
>> equivalent to doing byte-compilation, in which case it may make sense to
>> use the framework which already exists for that.
>>
>> Colin.
>> On 04/11/2025 01:18, EricT wrote:
>>
>> that is:
>>
>> if {[info exist ::cache($exp)]} {
>>     tailcall ::tcl::unsupported::assemble $::cache($exp)
>> }
>>
>> (hate gmail!)
>>
>>
>> On Mon, Nov 3, 2025 at 5:17 PM EricT <tw...@gm...> wrote:
>>
>>> and silly of me, it should be:
>>> if {[info exist ::cache($exp)]} {
>>>     tailcall ::tcl::unsupported::assemble $::cache($exp)
>>> }
>>>
>>>
>>> On Mon, Nov 3, 2025 at 4:50 PM EricT <tw...@gm...> wrote:
>>>
>>>> With a debug line back in plus the tailcall:
>>>>
>>>> proc = args {
>>>>     set exp [join $args]
>>>>     if { [info exist ::cache($exp)] } {
>>>>          return [tailcall  ::tcl::unsupported::assemble $::cache($exp)]
>>>>     }
>>>>     set tokens [tokenise $exp]
>>>>     deb1 "TOKENS = '$tokens'"
>>>>     set code [compile $tokens]
>>>>     deb1 "GENERATED CODE:\n$code\n"
>>>>     puts "exp= |$exp| code= |$code| ifexist: [info exist ::cache($exp)]"
>>>>     set ::cache($exp) $code
>>>>     uplevel [list ::tcl::unsupported::assemble $code]
>>>> }
>>>>
>>>> % set a 5
>>>> 5
>>>>  % set b 10
>>>> 10
>>>>  % = a + b
>>>> exp= |a + b| code= |push a; loadStk; push b; loadStk; add; | ifexist: 0
>>>> 15
>>>> % = a + b
>>>> 15
>>>>
>>>> % time {= a + b} 1000
>>>> 1.73 microseconds per iteration
>>>>
>>>>
>>>> Faster still!
>>>>
>>>> I thought the uplevel was needed to be able to get the local variables,
>>>> seems not.
>>>>
>>>>  % proc foo arg {set a 5; set b 10; set c [= a+b+arg]}
>>>>  % foo 5
>>>> exp= |a+b+arg| code= |push a; loadStk; push b; loadStk; add; push arg;
>>>> loadStk; add; | ifexist: 0
>>>> 20
>>>> % foo 5
>>>> 20
>>>>
>>>>  % proc foo arg {global xxx; set a 5; set b 10; set c [= a+b+arg+xxx]}
>>>>
>>>>  % set xxx 100
>>>> 100
>>>>  % foo 200
>>>> 315
>>>>  % time {foo 200} 10000
>>>> 2.1775 microseconds per iteration
>>>>
>>>>  % parray cache
>>>> cache(a + b)       = push a; loadStk; push b; loadStk; add;
>>>> cache(a+b+arg)     = push a; loadStk; push b; loadStk; add; push arg;
>>>> loadStk; add;
>>>> cache(a+b+arg+xxx) = push a; loadStk; push b; loadStk; add; push arg;
>>>> loadStk; add; push xxx; loadStk; add;
>>>>
>>>>
>>>> Very Impressive, great job Colin! Great catch Don!
>>>>
>>>> Eric
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Nov 3, 2025 at 4:22 PM Donald Porter via Tcl-Core <
>>>> tcl...@li...> wrote:
>>>>
>>>>> Check what effect replacing [uplevel] with [tailcall] has.
>>>>>
>>>>> On Nov 3, 2025, at 7:13 PM, EricT <tw...@gm...> wrote:
>>>>>
>>>>> Subject: Your bytecode expression evaluator - impressive results with
>>>>> caching!
>>>>>
>>>>> Hey Colin:
>>>>>
>>>>> I took a look at your bytecode-based expression evaluator and was
>>>>> intrigued by the approach. I made a small modification to add caching and
>>>>> the results are really impressive. Here's what I changed:
>>>>>
>>>>> proc = args {
>>>>>     set exp [join $args]
>>>>>     if {[info exist ::cache($exp)]} {
>>>>>         return [uplevel [list ::tcl::unsupported::assemble
>>>>> $::cache($exp)]]
>>>>>     }
>>>>>     set tokens [tokenise $exp]
>>>>>     deb1 "TOKENS = '$tokens'"
>>>>>     set code [compile $tokens]
>>>>>     deb1 "GENERATED CODE:\n$code\n"
>>>>>     set ::cache($exp) $code
>>>>>     uplevel [list ::tcl::unsupported::assemble $code]
>>>>> }
>>>>>
>>>>> The cache is just a simple array lookup - one line to store, one line
>>>>> to retrieve. But the performance impact is huge:
>>>>>
>>>>> Performance Tests
>>>>>
>>>>> Without caching
>>>>> % time {= 1 + 2} 1000
>>>>> 24.937 microseconds per iteration
>>>>>
>>>>> With caching
>>>>> % time {= 1 + 2} 1000
>>>>> 1.8 microseconds per iteration
>>>>>
>>>>> That's a 13x speedup! The tokenize and parse steps were eating about
>>>>> 92% of the execution time.
>>>>>
>>>>> The Real Magic: Bare Variables + Caching
>>>>>
>>>>> What really impressed me is how well your bare variable feature
>>>>> synergizes with caching:
>>>>>
>>>>> % set a 5
>>>>> 5
>>>>> % set b 6
>>>>> 6
>>>>> % = a + b
>>>>> 11
>>>>> % time {= a + b} 1000
>>>>> 2.079 microseconds per iteration
>>>>>
>>>>> Now change the variable values
>>>>> % set a 10
>>>>> 10
>>>>> % = a + b
>>>>> 16
>>>>> % time {= a + b} 1000
>>>>> 2.188 microseconds per iteration
>>>>>
>>>>> The cache entry stays valid even when the variable values change! Why?
>>>>> Because the bytecode stores variable names, not values:
>>>>>
>>>>> push a; loadStk; push b; loadStk; add;
>>>>>
>>>>> The loadStk instruction does runtime lookup, so:
>>>>> - Cache key is stable: "a + b"
>>>>> - Works for any values of a and b
>>>>> - One cache entry handles all value combinations
>>>>>
>>>>> Compare this to if we used $-substitution:
>>>>>
>>>>> = $a + $b    # With a=5, b=6 becomes "5 + 6"
>>>>> = $a + $b    # With a=10, b=6 becomes "10 + 6" - different cache key!
>>>>>
>>>>> Every value change would create a new cache entry or worse, a cache
>>>>> miss.
>>>>>
>>>>> Comparison to Other Approaches
>>>>>
>>>>> Tcl's expr:              about 0.40 microseconds
>>>>> Direct C evaluator:      about 0.53 microseconds
>>>>> Your cached approach:    about 1.80 microseconds
>>>>> Your uncached approach:  about 24.9 microseconds
>>>>>
>>>>> With caching, you're only 3-4x slower than a direct C evaluator.
>>>>>
>>>>>
>>>>> My Assessment
>>>>>
>>>>> Your design is excellent. The bare variable feature isn't just syntax
>>>>> sugar - it's essential for good cache performance. The synergy between:
>>>>>
>>>>> 1. Bare variables leading to stable cache keys
>>>>> 2. Runtime lookup keeping cache hot
>>>>> 3. Simple caching providing dramatic speedup
>>>>>
>>>>> makes this really elegant.
>>>>>
>>>>> My recommendation: Keep it in Tcl! The implementation is clean,
>>>>> performance is excellent (1.8 microseconds is plenty fast), and converting
>>>>> to C would add significant complexity for minimal gain (maybe getting to
>>>>> about 1.0 microseconds).
>>>>>
>>>>> The Tcl prototype with caching is actually the right solution here.
>>>>> Sometimes the prototype IS the product!
>>>>>
>>>>> Excellent work on this. The bytecode approach really shines with
>>>>> caching enabled.
>>>>>
>>>>> On Sun, Nov 2, 2025 at 10:14 AM Colin Macleod via Tcl-Core <
>>>>> tcl...@li...> wrote:
>>>>>
>>>>>> Hi again,
>>>>>>
>>>>>> I've now made a slightly more serious prototype, see
>>>>>> https://cmacleod.me.uk/tcl/expr_ng
>>>>>>
>>>>>> This is a modified version of the prototype I wrote for tip 676. It's
>>>>>> still in Tcl, but doesn't use `expr`.  It tokenises and parses the input,
>>>>>> then generates TAL bytecode and uses ::tcl::unsupported::assemble to run
>>>>>> that.  A few examples:
>>>>>>
>>>>>> (bin) 100 % set a [= 3.0/4]
>>>>>> 0.75
>>>>>> (bin) 101 % set b [= sin(a*10)]
>>>>>> 0.9379999767747389
>>>>>> (bin) 102 % set c [= (b-a)*100]
>>>>>> 18.79999767747389
>>>>>> (bin) 103 % namespace eval nn {set d [= 10**3]}
>>>>>> 1000
>>>>>> (bin) 104 % set e [= a?nn::d:b]
>>>>>> 1000
>>>>>> (bin) 105 % = {3 + [pwd]}
>>>>>> Calc: expected start of expression but found '[pwd]'
>>>>>> (bin) 106 % = {3 + $q}
>>>>>> Calc: expected start of expression but found '$q'
>>>>>> (bin) 107 % = sin (12)
>>>>>> -0.5365729180004349
>>>>>>
>>>>>> (bin) 108 % array set rr {one 1 two 2 three 3}
>>>>>> (bin) 110 % = a * rr(two)
>>>>>> Calc: expected operator but found '('
>>>>>> (bin) 111 % = a * $rr(two)
>>>>>> 1.5
>>>>>>
>>>>>> - You can use $ to get an array value substituted before the `=` code
>>>>>> sees the expression.
>>>>>>
>>>>>> (bin) 112 % string repeat ! [= nn::d / 15]
>>>>>> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>>>>>>
>>>>>> Colin.
>>>>>> On 02/11/2025 09:04, Donal Fellows wrote:
>>>>>>
>>>>>> Doing the job properly would definitely involve changing the
>>>>>> expression parser, with my suggested fix being to turn all bare words not
>>>>>> otherwise recognised as constants or in positions that look like function
>>>>>> calls (it's a parser with some lookahead) into simple variable reads (NB: C
>>>>>> resolves such ambiguities within itself differently, but that's one of the
>>>>>> nastiest parts of the language). We would need to retain $ support for
>>>>>> resolving ambiguity (e.g., array reads vs function calls; you can't safely
>>>>>> inspect the interpreter to resolve it at the time of compiling the
>>>>>> expression due to traces and unknown handlers) as well as compatibility,
>>>>>> but that's doable as it is a change only in cases that are currently errors.
>>>>>>
>>>>>> Adding assignment is quite a bit trickier, as that needs a new major
>>>>>> syntax class to describe the left side of the assignment. I suggest
>>>>>> omitting that from consideration at this stage.
>>>>>>
>>>>>> Donal.
>>>>>>
>>>>>> -------- Original message --------
>>>>>> From: Colin Macleod via Tcl-Core <tcl...@li...>
>>>>>> <tcl...@li...>
>>>>>> Date: 02/11/2025 08:13 (GMT+00:00)
>>>>>> To: Pietro Cerutti <ga...@ga...> <ga...@ga...>
>>>>>> Cc: tcl...@li..., av...@lo...
>>>>>> Subject: Re: [TCLCORE] Fwd: TIP 672 Implementation Complete - Ready
>>>>>> for Sponsorship
>>>>>>
>>>>>> Indeed, this toy implementation doesn't handle that:
>>>>>>
>>>>>> % = sin (12)
>>>>>> can't read "sin": no such variable
>>>>>>
>>>>>> I'm not sure that's serious, but it could be fixed in a C
>>>>>> implementation.
>>>>>>
>>>>>> _______________________________________________
>>>>>> Tcl-Core mailing list
>>>>>> Tcl...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/tcl-core
>>>>>>
>>>>> _______________________________________________
>>>>> Tcl-Core mailing list
>>>>> Tcl...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/tcl-core
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Tcl-Core mailing list
>>>>> Tcl...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/tcl-core
>>>>>
>>>>
>>
>> _______________________________________________
>> Tcl-Core mailing lis...@li...://lists.sourceforge.net/lists/listinfo/tcl-core
>>
>> _______________________________________________
>> Tcl-Core mailing list
>> Tcl...@li...
>> https://lists.sourceforge.net/lists/listinfo/tcl-core
>>
>