Menu
▾
▴
Re: [TCLCORE] [Ext] Re: [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)
|
From: Pietro C. <ga...@ga...> - 2025-11-07 09:16:38
|
<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto">On the other hand, the choice of implementation, be it C or Tcl, is quite orthogonal to the feature being in core, and thus from it requiring a TIP.<div><div><br id="lineBreakAtBeginningOfSignature"><div dir="ltr"><div>-- </div>Pietro Cerutti<div><span style="background-color: rgba(255, 255, 255, 0); font-size: 17pt;">I've pledged to give </span><span style="background-color: rgba(255, 255, 255, 0); font-size: 17pt;">10% of income to effective charities</span><span style="background-color: rgba(255, 255, 255, 0); font-size: 17pt;"> </span><span style="background-color: rgba(255, 255, 255, 0); font-size: 17pt;">and invite you to join me.</span></div><div>https://givingwhatwecan.org</div><div><br></div><div>Sent from a small device - please excuse brevity and typos.<br><div><br></div></div></div><div dir="ltr"><br><blockquote type="cite">On 7 Nov 2025, at 01:03, EricT <tw...@gm...> wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr"><div dir="ltr"><div><pre style="color:rgb(0,0,0);background-color:rgb(255,255,255);font-family:"Consolas";font-size:14pt;font-weight:normal;font-style:normal;text-decoration-line:none;white-space:pre-wrap">Hi Rene, I was experimenting with your TIP 674 prototype and realized both your let command and Colin's expression evaluator share something important: they could both be implemented in pure <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">Tcl</span> if <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">tcl</span>::unsupported::assemble were promoted to supported status with handle-based caching. This would avoid another round of "<span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">expr</span> wars" where lack of consensus leaves us with nothing. Instead of debating whose syntax is better, we'd each have the power to create our own Design-Specific Language. With <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">bytecode</span> caching, pure <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">Tcl</span> implementations achieve C extension performance - Colin's evaluator runs at 1.8 microseconds, nearly as fast as <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">expr</span>. Why does this matter for your TIP? If let performs just as well in <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">Tcl</span> as in C, why require a C implementation at all? C extensions take weeks to develop, need binaries for every platform/architecture combination, and ultimately call the same <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">bytecode</span> infrastructure that assemble uses. A supported <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">bytecode</span> <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">API</span> would make both proposals - and many others - viable without C. Perhaps we should advocate together for making the <span class="gmail-syntax9" style="text-decoration-line:underline;text-decoration-color:rgb(255,0,0);text-decoration-style:wavy">bytecode</span> compilation infrastructure officially supported, rather than competing for TIP approval on individual commands? Best, Eric</pre> <br></div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Wed, Nov 5, 2025 at 11:31 PM Zaumseil René via Tcl-Core <<a href="mailto:tcl...@li...">tcl...@li...</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg-4722256188258047040"> <div lang="DE-CH" style="overflow-wrap: break-word;"> <div class="m_-4722256188258047040WordSection1"> <p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Arial",sans-serif">Hello<u></u><u></u></span></p> <p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Arial",sans-serif"><u></u> <u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Arial",sans-serif">I also like the = approach as a new expr command without need of $ for variables.<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Arial",sans-serif">But I'm not sure about the currently proposed syntax.<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Arial",sans-serif">Is it "= 1 + 2" or "= 1+2" (with or without spaces)?<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Arial",sans-serif">If it is the later then I would propose to use some of the syntax of the "let" from tip 674.<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Arial",sans-serif"><u></u> <u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Arial",sans-serif">Some syntax ideas:<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Arial",sans-serif">"= 1+2" return 3, simple case<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Arial",sans-serif">"= a 1+2 b 2+4 c a+b" return {3 6 9} and set a,b,c<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Arial",sans-serif">"= 1+2 = 2+4" return {3 6}<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Arial",sans-serif">And may be some others…<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Arial",sans-serif"><u></u> <u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Arial",sans-serif">Regards<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Arial",sans-serif">rene<u></u><u></u></span></p> <div style="border-width:1pt medium medium;border-style:solid none none;border-color:rgb(225,225,225) currentcolor currentcolor;padding:3pt 0cm 0cm"> <p class="MsoNormal"><b><span lang="DE" style="font-size:11pt;font-family:"Calibri",sans-serif">Von:</span></b><span lang="DE" style="font-size:11pt;font-family:"Calibri",sans-serif"> EricT <<a href="mailto:tw...@gm..." target="_blank">tw...@gm...</a>> <br> <b>Gesendet:</b> Mittwoch, 5. November 2025 23:22<br> <b>An:</b> Colin Macleod <<a href="mailto:col...@ya..." target="_blank">col...@ya...</a>>; <a href="mailto:tcl...@li..." target="_blank">tcl...@li...</a><br> <b>Betreff:</b> [Ext] Re: [TCLCORE] [=] for concise expressions (was Re: TIP 672 Implementation Complete - Ready for Sponsorship)<u></u><u></u></span></p> </div> <p class="MsoNormal"><u></u> <u></u></p> <div> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">Hi Colin,<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u> <u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">I've successfully modified your amazing code to handle arrays. In doing so, I also found 2 other issues, one is with your Boolean check, the other with your function name check, both because of [string is] issues. <u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u> <u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">- Boolean check: `$token <span class="m_-4722256188258047040gmail-syntax9">eq</span> "false" || $token <span class="m_-4722256188258047040gmail-syntax9">eq</span> "true"` (was `[string is <span class="m_-4722256188258047040gmail-syntax9">boolean</span> $token]` - treated 'f','n', 't', 'y', <span class="m_-4722256188258047040gmail-syntax9">etc</span>. variables as <span class="m_-4722256188258047040gmail-syntax9">boolean</span> false, no, true, yes, ...)<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u> <u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">- Function check: `[regexp {^[[:alpha:]]} $token]` (was `[string is alpha $token]` - broke log10, atan2)<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u> <u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u> <u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">here's the code for arrays:<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u> <u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> # Function call or array reference?<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> set <span class="m_-4722256188258047040gmail-syntax9">nexttok</span> [<span class="m_-4722256188258047040gmail-syntax9">lindex</span> $::tokens $::<span class="m_-4722256188258047040gmail-syntax9">tokpos</span>]<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> if {$<span class="m_-4722256188258047040gmail-syntax9">nexttok</span> <span class="m_-4722256188258047040gmail-syntax9">eq</span> "(" && [regexp {^[[:alpha:]]} $token]} {<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> set fun [<span class="m_-4722256188258047040gmail-syntax9">namespace</span> which <span class="m_-4722256188258047040gmail-syntax9">tcl</span>::<span class="m_-4722256188258047040gmail-syntax9">mathfunc</span>::$token]<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> if {$fun ne {}} {<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> # It's a function<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> <span class="m_-4722256188258047040gmail-syntax9">incr</span> ::<span class="m_-4722256188258047040gmail-syntax9">tokpos</span><u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> set <span class="m_-4722256188258047040gmail-syntax9">opcodes</span> "push $fun; "<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> append <span class="m_-4722256188258047040gmail-syntax9">opcodes</span> [<span class="m_-4722256188258047040gmail-syntax9">parseFuncArgs</span>]<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> return $<span class="m_-4722256188258047040gmail-syntax9">opcodes</span><u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> } else {<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> # Not a function, assume array reference<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> <span class="m_-4722256188258047040gmail-syntax9">incr</span> ::<span class="m_-4722256188258047040gmail-syntax9">tokpos</span><u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> set <span class="m_-4722256188258047040gmail-syntax9">opcodes</span> "push $token; "<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> # Parse the index expression - leaves VALUE on stack<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> append <span class="m_-4722256188258047040gmail-syntax9">opcodes</span> [parse 0]<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> # Expect closing <span class="m_-4722256188258047040gmail-syntax9">paren</span><u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> set closing [<span class="m_-4722256188258047040gmail-syntax9">lindex</span> $::tokens $::<span class="m_-4722256188258047040gmail-syntax9">tokpos</span>]<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> if {$closing ne ")"} {<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> error "<span class="m_-4722256188258047040gmail-syntax9">Calc</span>: expected ')' but found '$closing'"<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> }<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> # Stack now has: [<span class="m_-4722256188258047040gmail-syntax9">arrayname</span>, <span class="m_-4722256188258047040gmail-syntax9">indexvalue</span>]<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> <span class="m_-4722256188258047040gmail-syntax9">incr</span> ::<span class="m_-4722256188258047040gmail-syntax9">tokpos</span> <u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> append <span class="m_-4722256188258047040gmail-syntax9">opcodes</span> "<span class="m_-4722256188258047040gmail-syntax9">loadArrayStk</span>; "<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> return $<span class="m_-4722256188258047040gmail-syntax9">opcodes</span><u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> }<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"> }<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u> <u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u> <u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">In addition, there has indeed been some changes in the <span class="m_-4722256188258047040gmail-syntax9">bytecode</span>, land and <span class="m_-4722256188258047040gmail-syntax9">lor</span> are no longer supported in 9.0 although they work in 8.6.<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u> <u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">I had an AI generate some 117 test cases, which all pass on 8.6 and 111 on 9.x (the land/<span class="m_-4722256188258047040gmail-syntax9">lor</span> not being tested in 9.x).<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u> <u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">Colin, with your permission, I can post the code as a new file, with all the test cases, say on a repository at <span class="m_-4722256188258047040gmail-syntax9">github</span>.<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u> <u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">I think a new TIP is worth considering; one that promotes assemble to a supported form, with a compile and handle approach to avoid the time parsing the <span class="m_-4722256188258047040gmail-syntax9">ascii</span> byte code text. I think that this would be great for your = command, but also quite useful for others who might want to create their own little languages.<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u> <u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">By doing it this way, it remains pure <span class="m_-4722256188258047040gmail-syntax9">tcl</span>, and avoids all the problems with different systems and hardware that a binary extension would create. In the end, I believe your code can achieve performance parity with expr. Not only does it remove half the [expr {...}] baggage, but all the $'s too! So much easier on these old eyes.<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u> <u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">Regards,<u></u><u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black"><u></u> <u></u></span></pre> <pre style="background:white"><span style="font-size:14pt;font-family:Consolas;color:black">Eric<u></u><u></u></span></pre> <p class="MsoNormal"><u></u> <u></u></p> </div> <p class="MsoNormal"><u></u> <u></u></p> <div> <div> <p class="MsoNormal">On Tue, Nov 4, 2025 at 1:06<span style="font-family:"Arial",sans-serif"> </span>PM EricT <<a href="mailto:tw...@gm..." target="_blank">tw...@gm...</a>> wrote:<u></u><u></u></p> </div> <blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:currentcolor currentcolor currentcolor rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm"> <div> <div> <p class="MsoNormal">Hi Colin,<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">Hmmm, why can't you do bareword on $a(b) as a(b) you just need to do an uplevel to see if a is a variable, if not, it would have to be a function. True?<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">% tcl::unsupported::disassemble script {set a [expr {$b($c)}] }<br> snip<u></u><u></u></p> </div> <div> <p class="MsoNormal" style="margin-bottom:12pt"> Command 2: "expr {$b($c)}..."<br> (2) push1 1 # "b"<br> (4) push1 2 # "c"<br> (6) loadStk <br> (7) loadArrayStk <br> (8) tryCvtToNumeric <br> (9) storeStk <br> (10) done <u></u><u></u></p> </div> <div> <p class="MsoNormal">This doesn't look too much different from what you are producing.<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">I think what's really needed here is a TIP that would open up the bytecode a bit so you don't need to use an unsupported command. And then maybe even have a new command to take the string byte code you are now producing and return a handle to a cached version that was probably equivalent to the existing bytecode. Then your cache array would be<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal"> set cache($exp) $handle<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">Instead of it having to parse the text, it could be as fast as bytecode. You'd likely be just as fast as expr, and safe as well, since you can't pass a string command in where the bareword is required:<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal" style="margin-bottom:12pt"> % set x {[pwd]}<br> [pwd]<br> % = sqrt(x)<br> exp= |sqrt(x)| code= |push ::tcl::mathfunc::sqrt; push x; loadStk; invokeStk 2; | ifexist: 0<br> expected floating-point number but got "[pwd]"<u></u><u></u></p> </div> <div> <p class="MsoNormal">I think you really have something here, perhaps this is the best answer yet to slay the expr dragon! <u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">Regards,<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">Eric<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> </div> <p class="MsoNormal"><u></u> <u></u></p> <div> <div> <p class="MsoNormal">On Tue, Nov 4, 2025 at 6:52<span style="font-family:"Arial",sans-serif"> </span>AM Colin Macleod via Tcl-Core <<a href="mailto:tcl...@li..." target="_blank">tcl...@li...</a>> wrote:<u></u><u></u></p> </div> <blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:currentcolor currentcolor currentcolor rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm"> <div> <p>Hi Eric,<br> <br> That's very neat! <br> <br> Yes, a pure Tcl version could go into TclLib. I still think it may be worth trying a C implementation though. The work-around that's needed for array references [= 2* $a(b)] would defeat the caching, so it would be good to speed up the parsing if possible. Also I think your caching may be equivalent to doing byte-compilation, in which case it may make sense to use the framework which already exists for that.<br> <br> Colin.<u></u><u></u></p> <div> <p class="MsoNormal">On 04/11/2025 01:18, EricT wrote:<u></u><u></u></p> </div> <blockquote style="margin-top:5pt;margin-bottom:5pt"> <div> <div> <p class="MsoNormal">that is:<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">if {[info exist ::cache($exp)]} { <u></u><u></u></p> </div> <p class="MsoNormal"> tailcall ::tcl::unsupported::assemble $::cache($exp) <br> } <u></u><u></u></p> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">(hate gmail!)<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> </div> <p class="MsoNormal"><u></u> <u></u></p> <div> <div> <p class="MsoNormal">On Mon, Nov 3, 2025 at 5:17<span style="font-family:"Arial",sans-serif"> </span>PM EricT <<a href="mailto:tw...@gm..." target="_blank">tw...@gm...</a>> wrote:<u></u><u></u></p> </div> <blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:currentcolor currentcolor currentcolor rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm"> <div> <p class="MsoNormal" style="margin-bottom:12pt">and silly of me, it should be:<br> if {[info exist ::cache($exp)]} { <br> tailcall ::tcl::unsupported::assemble $::cache($exp) <br> } <u></u><u></u></p> </div> <p class="MsoNormal"><u></u> <u></u></p> <div> <div> <p class="MsoNormal">On Mon, Nov 3, 2025 at 4:50<span style="font-family:"Arial",sans-serif"> </span>PM EricT <<a href="mailto:tw...@gm..." target="_blank">tw...@gm...</a>> wrote:<u></u><u></u></p> </div> <blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:currentcolor currentcolor currentcolor rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm"> <div> <div> <p class="MsoNormal">With a debug line back in plus the tailcall:<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">proc = args {<u></u><u></u></p> </div> <p class="MsoNormal"> set exp [join $args]<br> if { [info exist ::cache($exp)] } {<br> return [tailcall ::tcl::unsupported::assemble $::cache($exp)]<br> }<br> set tokens [tokenise $exp]<br> deb1 "TOKENS = '$tokens'"<br> set code [compile $tokens]<br> deb1 "GENERATED CODE:\n$code\n" <br> puts "exp= |$exp| code= |$code| ifexist: [info exist ::cache($exp)]"<br> set ::cache($exp) $code<br> uplevel [list ::tcl::unsupported::assemble $code]<br> }<u></u><u></u></p> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal" style="margin-bottom:12pt">% set a 5<br> 5<br> % set b 10<br> 10<br> % = a + b<br> exp= |a + b| code= |push a; loadStk; push b; loadStk; add; | ifexist: 0<br> 15<br> % = a + b<br> 15<br> <br> % time {= a + b} 1000<br> 1.73 microseconds per iteration<br> <br> <u></u><u></u></p> </div> <div> <p class="MsoNormal">Faster still!<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">I thought the uplevel was needed to be able to get the local variables, seems not.<br> <br> % proc foo arg {set a 5; set b 10; set c [= a+b+arg]}<br> % foo 5<br> exp= |a+b+arg| code= |push a; loadStk; push b; loadStk; add; push arg; loadStk; add; | ifexist: 0<br> 20<br> % foo 5<br> 20<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal"> % proc foo arg {global xxx; set a 5; set b 10; set c [= a+b+arg+xxx]}<br> <br> % set xxx 100<br> 100<br> % foo 200<br> 315<br> % time {foo 200} 10000<br> 2.1775 microseconds per iteration<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal"> % parray cache<u></u><u></u></p> </div> <div> <p class="MsoNormal" style="margin-bottom:12pt">cache(a + b) = push a; loadStk; push b; loadStk; add; <br> cache(a+b+arg) = push a; loadStk; push b; loadStk; add; push arg; loadStk; add; <br> cache(a+b+arg+xxx) = push a; loadStk; push b; loadStk; add; push arg; loadStk; add; push xxx; loadStk; add; <u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">Very Impressive, great job Colin! Great catch Don!<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">Eric<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> </div> <p class="MsoNormal"><u></u> <u></u></p> <div> <div> <p class="MsoNormal">On Mon, Nov 3, 2025 at 4:22<span style="font-family:"Arial",sans-serif"> </span>PM Donald Porter via Tcl-Core <<a href="mailto:tcl...@li..." target="_blank">tcl...@li...</a>> wrote:<u></u><u></u></p> </div> <blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:currentcolor currentcolor currentcolor rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm"> <div> <p class="MsoNormal">Check what effect replacing [uplevel] with [tailcall] has.<u></u><u></u></p> <div> <p class="MsoNormal"><br> <br> <u></u><u></u></p> <blockquote style="margin-top:5pt;margin-bottom:5pt"> <div> <p class="MsoNormal">On Nov 3, 2025, at 7:13<span style="font-family:"Arial",sans-serif"> </span>PM, EricT <<a href="mailto:tw...@gm..." target="_blank">tw...@gm...</a>> wrote:<u></u><u></u></p> </div> <p class="MsoNormal"><u></u> <u></u></p> <div> <div> <p class="MsoNormal">Subject: Your bytecode expression evaluator - impressive results with caching!<br> <br> Hey Colin:<br> <br> I took a look at your bytecode-based expression evaluator and was intrigued by the approach. I made a small modification to add caching and the results are really impressive. Here's what I changed:<br> <br> proc = args {<br> set exp [join $args]<br> if {[info exist ::cache($exp)]} {<br> return [uplevel [list ::tcl::unsupported::assemble $::cache($exp)]]<br> }<br> set tokens [tokenise $exp]<br> deb1 "TOKENS = '$tokens'"<br> set code [compile $tokens]<br> deb1 "GENERATED CODE:\n$code\n" <br> set ::cache($exp) $code<br> uplevel [list ::tcl::unsupported::assemble $code]<br> }<br> <br> The cache is just a simple array lookup - one line to store, one line to retrieve. But the performance impact is huge:<br> <br> Performance Tests<br> <br> Without caching<br> % time {= 1 + 2} 1000<br> 24.937 microseconds per iteration<br> <br> With caching<br> % time {= 1 + 2} 1000<br> 1.8 microseconds per iteration<br> <br> That's a 13x speedup! The tokenize and parse steps were eating about 92% of the execution time.<br> <br> The Real Magic: Bare Variables + Caching<br> <br> What really impressed me is how well your bare variable feature synergizes with caching:<br> <br> % set a 5<br> 5<br> % set b 6 <br> 6<br> % = a + b<br> 11<br> % time {= a + b} 1000<br> 2.079 microseconds per iteration<br> <br> Now change the variable values<br> % set a 10<br> 10<br> % = a + b<br> 16<br> % time {= a + b} 1000<br> 2.188 microseconds per iteration<br> <br> The cache entry stays valid even when the variable values change! Why? Because the bytecode stores variable names, not values:<br> <br> push a; loadStk; push b; loadStk; add;<br> <br> The loadStk instruction does runtime lookup, so:<br> - Cache key is stable: "a + b"<br> - Works for any values of a and b<br> - One cache entry handles all value combinations<br> <br> Compare this to if we used $-substitution:<br> <br> = $a + $b # With a=5, b=6 becomes "5 + 6"<br> = $a + $b # With a=10, b=6 becomes "10 + 6" - different cache key!<br> <br> Every value change would create a new cache entry or worse, a cache miss.<br> <br> Comparison to Other Approaches<br> <br> Tcl's expr: about 0.40 microseconds<br> Direct C evaluator: about 0.53 microseconds <br> Your cached approach: about 1.80 microseconds<br> Your uncached approach: about 24.9 microseconds<br> <br> With caching, you're only 3-4x slower than a direct C evaluator.<br> <br> <br> My Assessment<br> <br> Your design is excellent. The bare variable feature isn't just syntax sugar - it's essential for good cache performance. The synergy between:<br> <br> 1. Bare variables leading to stable cache keys<br> 2. Runtime lookup keeping cache hot<br> 3. Simple caching providing dramatic speedup<br> <br> makes this really elegant.<br> <br> My recommendation: Keep it in Tcl! The implementation is clean, performance is excellent (1.8 microseconds is plenty fast), and converting to C would add significant complexity for minimal gain (maybe getting to about 1.0 microseconds).<br> <br> The Tcl prototype with caching is actually the right solution here. Sometimes the prototype IS the product!<br> <br> Excellent work on this. The bytecode approach really shines with caching enabled.<u></u><u></u></p> </div> <p class="MsoNormal"><u></u> <u></u></p> <div> <div> <p class="MsoNormal">On Sun, Nov 2, 2025 at 10:14<span style="font-family:"Arial",sans-serif"> </span>AM Colin Macleod via Tcl-Core <<a href="mailto:tcl...@li..." target="_blank">tcl...@li...</a>> wrote:<u></u><u></u></p> </div> <blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:currentcolor currentcolor currentcolor rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm"> <div> <p>Hi again,<u></u><u></u></p> <p>I've now made a slightly more serious prototype, see <a href="https://cmacleod.me.uk/tcl/expr_ng" target="_blank"> https://cmacleod.me.uk/tcl/expr_ng</a><u></u><u></u></p> <p>This is a modified version of the prototype I wrote for tip 676. It's still in Tcl, but doesn't use `expr`. It tokenises and parses the input, then generates TAL bytecode and uses ::tcl::unsupported::assemble to run that. A few examples:<u></u><u></u></p> <blockquote style="margin-top:5pt;margin-bottom:5pt"> <p><span style="color:blue">(bin) 100 % set a [= 3.0/4]<br> 0.75<br> (bin) 101 % set b [= sin(a*10)]<br> 0.9379999767747389<br> (bin) 102 % set c [= (b-a)*100]<br> 18.79999767747389<br> (bin) 103 % namespace eval nn {set d [= 10**3]}<br> 1000<br> (bin) 104 % set e [= a?nn::d:b]<br> 1000<br> (bin) 105 % = {3 + [pwd]}<br> Calc: expected start of expression but found '[pwd]'<br> (bin) 106 % = {3 + $q}<br> Calc: expected start of expression but found '$q'<br> (bin) 107 % = sin (12)<br> -0.5365729180004349</span><u></u><u></u></p> <p><span style="color:blue">(bin) 108 % array set rr {one 1 two 2 three 3}<br> (bin) 110 % = a * rr(two)<br> Calc: expected operator but found '('<br> (bin) 111 % = a * $rr(two)<br> 1.5</span><u></u><u></u></p> </blockquote> <p>- You can use $ to get an array value substituted before the `=` code sees the expression.<u></u><u></u></p> <blockquote style="margin-top:5pt;margin-bottom:5pt"> <p><span style="color:blue">(bin) 112 % string repeat ! [= nn::d / 15]<br> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!</span><u></u><u></u></p> </blockquote> <p>Colin.<u></u><u></u></p> <div> <p class="MsoNormal">On 02/11/2025 09:04, Donal Fellows wrote:<u></u><u></u></p> </div> <blockquote style="margin-top:5pt;margin-bottom:5pt"> <div> <p class="MsoNormal">Doing the job properly would definitely involve changing the expression parser, with my suggested fix being to turn all bare words not otherwise recognised as constants or in positions that look like function calls (it's a parser with some lookahead) into simple variable reads (NB: C resolves such ambiguities within itself differently, but that's one of the nastiest parts of the language). We would need to retain $ support for resolving ambiguity (e.g., array reads vs function calls; you can't safely inspect the interpreter to resolve it at the time of compiling the expression due to traces and unknown handlers) as well as compatibility, but that's doable as it is a change only in cases that are currently errors.<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">Adding assignment is quite a bit trickier, as that needs a new major syntax class to describe the left side of the assignment. I suggest omitting that from consideration at this stage.<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">Donal.<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <div> <p class="MsoNormal">-------- Original message --------<u></u><u></u></p> </div> <div> <p class="MsoNormal">From: Colin Macleod via Tcl-Core <a href="mailto:tcl...@li..." target="_blank"> <tcl...@li...></a> <u></u><u></u></p> </div> <div> <p class="MsoNormal">Date: 02/11/2025 08:13 (GMT+00:00) <u></u><u></u></p> </div> <div> <p class="MsoNormal">To: Pietro Cerutti <a href="mailto:ga...@ga..." target="_blank"> <ga...@ga...></a> <u></u><u></u></p> </div> <div> <p class="MsoNormal">Cc: <a href="mailto:tcl...@li..." target="_blank"> tcl...@li...</a>, <a href="mailto:av...@lo..." target="_blank"> av...@lo...</a> <u></u><u></u></p> </div> <div> <p class="MsoNormal">Subject: Re: [TCLCORE] Fwd: TIP 672 Implementation Complete - Ready for Sponsorship <u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u> <u></u></p> </div> <p>Indeed, this toy implementation doesn't handle that:<u></u><u></u></p> <blockquote style="margin-top:5pt;margin-bottom:5pt"> <p><span style="color:blue">% = sin (12)<br> can't read "sin": no such variable</span><u></u><u></u></p> </blockquote> <p>I'm not sure that's serious, but it could be fixed in a C implementation.<u></u><u></u></p> </blockquote> </div> <p class="MsoNormal">_______________________________________________<br> Tcl-Core mailing list<br> <a href="mailto:Tcl...@li..." target="_blank">Tcl...@li...</a><br> <a href="https://lists.sourceforge.net/lists/listinfo/tcl-core" target="_blank">https://lists.sourceforge.net/lists/listinfo/tcl-core</a><u></u><u></u></p> </blockquote> </div> <p class="MsoNormal">_______________________________________________<br> Tcl-Core mailing list<br> <a href="mailto:Tcl...@li..." target="_blank">Tcl...@li...</a><br> <a href="https://lists.sourceforge.net/lists/listinfo/tcl-core" target="_blank">https://lists.sourceforge.net/lists/listinfo/tcl-core</a><u></u><u></u></p> </div> </blockquote> </div> <p class="MsoNormal"><u></u> <u></u></p> </div> <p class="MsoNormal">_______________________________________________<br> Tcl-Core mailing list<br> <a href="mailto:Tcl...@li..." target="_blank">Tcl...@li...</a><br> <a href="https://lists.sourceforge.net/lists/listinfo/tcl-core" target="_blank">https://lists.sourceforge.net/lists/listinfo/tcl-core</a><u></u><u></u></p> </blockquote> </div> </blockquote> </div> </blockquote> </div> <p class="MsoNormal" style="margin-bottom:12pt"><u></u> <u></u></p> <pre>_______________________________________________<u></u><u></u></pre> <pre>Tcl-Core mailing list<u></u><u></u></pre> <pre><a href="mailto:Tcl...@li..." target="_blank">Tcl...@li...</a><u></u><u></u></pre> <pre><a href="https://lists.sourceforge.net/lists/listinfo/tcl-core" target="_blank">https://lists.sourceforge.net/lists/listinfo/tcl-core</a><u></u><u></u></pre> </blockquote> </div> <p class="MsoNormal">_______________________________________________<br> Tcl-Core mailing list<br> <a href="mailto:Tcl...@li..." target="_blank">Tcl...@li...</a><br> <a href="https://lists.sourceforge.net/lists/listinfo/tcl-core" target="_blank">https://lists.sourceforge.net/lists/listinfo/tcl-core</a><u></u><u></u></p> </blockquote> </div> </blockquote> </div> </div> </div> _______________________________________________<br> Tcl-Core mailing list<br> <a href="mailto:Tcl...@li..." target="_blank">Tcl...@li...</a><br> <a href="https://lists.sourceforge.net/lists/listinfo/tcl-core" rel="noreferrer" target="_blank">https://lists.sourceforge.net/lists/listinfo/tcl-core</a><br> </div></blockquote></div> <span>_______________________________________________</span><br><span>Tcl-Core mailing list</span><br><span>Tcl...@li...</span><br><span>https://lists.sourceforge.net/lists/listinfo/tcl-core</span><br></div></blockquote></div></div></body></html> |
