Menu
▾
▴
Re: [TCLCORE] Prototype Implementation of TIP 672 - $(expr) Syntax
|
From: Donal F. <don...@ma...> - 2025-10-23 11:16:18
|
[Hmm, looks like I forget to send this...]
My point was severalfold:
1.
The shell syntax had double-parenthesis.
2.
A search for that exact syntax in Tcl code yielded almost no valid results; only JimTcl code appears to be currently affected. The search was only of code that's public, but that acts as a sampler for the space of non-public code.
3.
The syntax $((...)) is therefore concluded to be available whereas the shorter $(...) syntax is contested; we're talking about two or three orders of magnitude more instances. That's going to be significant migration trouble. Nobody appears to be using literal array keys that start and end with parentheses in practice.
*
The cases where they're using keys which contain them and are using an empty-named array don't count; the syntax must be starting with exactly $(( or $( for the syntax to trigger
*
Updating the implementation proposed syntax (plus the TIP) to $((...)) is less work than tracking down all the places broken by sticking to $(...).
4.
$((...)) is still nicer and shorter and easier to type than [expr {...}]; the syntax feature will likely reduce bugs.
One doesn't always get exactly what one wants. You've won almost all the arguments (and easily) except for the one about the syntax, where it turns out it's being used in practice for other purposes. In many places. This is where my TCT hat gets involved: I have to consider all that other code as well as your proposal.
What follows will be heated discussion. Syntax discussions have historically always been so.
Donal.
________________________________
From: EricT <tw...@gm...>
Sent: Tuesday, October 21, 2025 13:36
To: Donal Fellows <don...@ma...>
Cc: Brian Griffin <bri...@ea...>; tc...@ro... <tc...@ro...>; Tcl Core List <tcl...@li...>
Subject: Re: [TCLCORE] Prototype Implementation of TIP 672 - $(expr) Syntax
Hi Donal:
Yes, I should have been more careful with that. I see that it is actually $((...)) in bash.
I think the best approach is to find all the uses by both searching and also a pragma that optionally will cause any use of $() to be flagged with an error. Then convert to using one of the two alternative methods.
I understand empty array names exist in production code. It's also important to note that ${()} doesn't work for all cases - it only handles literal indices. When the index involves variable substitution (like $($var)), users must use [set (...)] instead.
The migration path is clear I believe: use [set (...)] which works for all cases, both before and after the feature is enabled.
Thanks again for the feedback.
Eric
On Tue, Oct 21, 2025 at 1:55 AM Donal Fellows <don...@ma...<mailto:don...@ma...>> wrote:
EricT2025-10-21 04:12
1. Shell precedent: The $(command) syntax is well-established in bash/sh for command substitution. Tcl users familiar with shell scripting would find this natural, not confusing.
Surely the better shell precedent would be the $((expression)) form? That would also be quite a bit less likely to clash with existing use cases. Hmm, a quick search with https://github.com/search?q=%24%28%28+language%3ATcl&type=code [github.com]<https://urldefense.com/v3/__https://github.com/search?q=*24*28*28*language*3ATcl&type=code__;JSUlKyU!!PDiH4ENfjr2_Jw!Eot1BTBAjUrPeBxEg8km_8tqRFXpHzfHkiNZX7qagLMYARtCjwfj3_uX-YkRBZbGR4sbbT_P-u3u9zqo38ySjFCk4pY$> gives 29 hits (most of which appear to be false positives; the real hits seem to be from one place in JimTcl itself), as opposed to 4k hits when looking for $( in Tcl code; two orders of magnitude less usage. One of the really nice things about a resource like GitHub is that one can quickly search for language syntax features across a lot of code and get a good feeling for practical compatibility.
While $(([llength $x] - 17)) is no shorter than {=}{[llength $x] - 17}, it's definitely easier to type!
(Eric: Don't lose heart! These discussions are a lot less acrimonious than the discussion over what became {*} was.)
Donal.
________________________________
From: EricT <tw...@gm...<mailto:tw...@gm...>>
Sent: Tuesday, October 21, 2025 04:11
To: Brian Griffin <bri...@ea...<mailto:bri...@ea...>>
Cc: tc...@ro...<mailto:tc...@ro...> <tc...@ro...<mailto:tc...@ro...>>; Tcl Core List <tcl...@li...<mailto:tcl...@li...>>
Subject: Re: [TCLCORE] Prototype Implementation of TIP 672 - $(expr) Syntax
Thanks for the feedback, Brian! I understand your concern about the semantic distinction between $ (value) and [ ] (execution).
I'd offer a few thoughts:
1. Shell precedent: The $(command) syntax is well-established in bash/sh for command substitution. Tcl users familiar with shell scripting would find this natural, not confusing.
2. $ already involves execution: Even $var involves command execution internally (TclGetVar), it's just optimized. The distinction between "value substitution" and "command execution" is somewhat artificial at the implementation level.
3. expr is special and has historical precedent: Unlike arbitrary commands, expr is used so frequently that syntactic sugar makes sense - similar to how $var is sugar for [set var]. We optimize the common case. I believe there was a time in early Tcl before $var existed when [set var] was the only choice - if so, adding $var as syntactic sugar for variable substitution was a usability win. $(expr) follows the same pattern - sugar for the extremely common [expr {...}] case.
4. Consistency: $(expr) fits the pattern of "$ means substitute something here" - whether a variable, array element, or expression result. Modern languages like Python have embraced similar concepts with f-strings that allow {expression} for inline evaluation. $(expr) brings this convenience to Tcl while maintaining our substitution semantics.
That said, I appreciate the philosophical consistency argument. Do you see the security benefits (auto-bracing) as compelling enough to outweigh the semantic concern?
Did you see the email I sent you about lseq? Maybe it landed in your spam folder - that happened once before as I recall.
Eric
On Mon, Oct 20, 2025 at 7:19 PM Brian Griffin <bri...@ea...<mailto:bri...@ea...>> wrote:
I like the idea of simplifying expression substitutions. The problem I have is that the '$' is defined to be a value substitution. Running a command (execution) is a [] substitution, not a '$' substitution. Conflating the two modes of substitution can cause confusion for newbies, and even experienced programmers.
I recognize that this view is a very subtle distinction, but I think it's important.
Think of it like conflating * vs & in C. (Maybe not that bad, but similar)
-Brian
On Oct 20, 2025, at 07:22, EricT <tw...@gm...<mailto:tw...@gm...>> wrote:
Regarding the discussion in the telco today. If the incompatibility discussed is the known empty array issue, then for that I would propose the following:
There could be a pragma or tcl_dollar_expr global variable with several settings.
1. Work in compatibility mode, i.e. as though the feature was not implemented at all.
2. Work in diagnostic mode, throw an error for any $(...) use
3. Work in full implementation mode
The code to do this, given a global C variable to check would be trivial.
This value could default to 1. for at least 9.1 or 9.2 but there could be an option, similar to tclsh -encoding, that would allow the pragma variable to be set before any init script code is run. This would give developers the opportunity to set the variable to mode 2 in order to easily track down uses of $(index) before any code were to begin using $(expr).
I think it would be quite rare if this syntax change would affect binary extensions, so it should be only with script level coding.
The fix to any code that is using the empty array variable, is to change from,
set val $(index)
set val $($var) - or any other valid index string
to
set val ${(index)}
set val ${($var)}
Likewise for any other use of the $(...) substitution in current use.
This change can begin to be made right away, since it is a compatible syntax that does not change the semantics. Once all the init time script code or packages that use the un-braced syntax are found and changed, the feature could begin to be used by those who are interested.
On the other hand, if the incompatibility is something other than this empty array, I'd be most appreciative to know what it is.
thanks
Eric
On Sun, Oct 19, 2025 at 3:44 AM EricT <tw...@gm...<mailto:tw...@gm...>> wrote:
Harald,
Thank you for your willingness to sponsor!
To clarify: I am not the author of Jim Tcl - I'm just a long-time Tcl user who, in retirement, wanted to give something back to the Tcl community. Jim Tcl's successful use of this syntax for years demonstrates that the concept is viable.
Regarding the telco: I attempted to join previously but was unsuccessful with the setup. I won't be able to participate on Monday, but I'm available via email for any questions or clarifications that arise from the discussion.
I don't currently have write access to either the Tcl Fossil repository or the TIP repository, so I'm unable to make changes directly. At nearly 80 years old, I'm not comfortable learning new version control systems on my own. If the TIP moves forward, I'd need guidance and assistance with the Fossil workflow, or perhaps someone from the core team who shares our interest in this TIP could handle the integration based on my GitHub prototype.
I'm currently developing a comprehensive test suite in standard tcltest format, with particular focus on edge cases.
Looking forward to hearing how Monday's discussion goes!
Best regards,
Eric
On Sun, Oct 19, 2025 at 2:05 AM Harald Oehlmann <har...@el...<mailto:har...@el...>> wrote:
Eric,
sponsoring is no problem. The point is to get positive votes.
This requires a positive discussion.
You are the Author of Jim?
Could you participate on the telco on Monday?
Do you have write access to the tcl fossil? We would need the
implementation there.
Thanks for all,
Harald
Am 18.10.2025 um 22:58 schrieb EricT:
> Thank you for the positive feedback and for raising this in Monday's
> telco! I'm encouraged by your support.
>
> Regarding $(a) - you're right that reading an empty array element with a
> variable index is a valid construct. However, this is explicitly
> addressed in the TIP and the repository README. The workarounds are
> straightforward:
>
> ${(a)} # Braced form
> [set (a)] # Command substitution
>
> Both still work. In fact, code using $(varname) could be proactively
> modified to use ${(varname)} to indicate the clear intent of an empty
> array reference, which improves readability. The security, performance,
> and usability benefits of $(...) seemed to justify this trade-off for
> Tcl 9.x where some incompatibilities are expected.
>
> Given your interest in the feature, would you be willing to consider
> sponsoring TIP 672? The implementation is working and minimal (~100
> lines across two files), with the main open question being the preferred
> approach for tracking synthetic strings for cleanup. Your guidance on
> that architectural decision would be particularly valuable.
>
> The prototype repository with full implementation and examples is here:
>
> https://github.com/rocketship88/tcl-tip-672-prototype [github.com]<https://urldefense.com/v3/__https://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8buJgHl4$> <https://
> github.com/rocketship88/tcl-tip-672-prototype [github.com]<https://urldefense.com/v3/__http://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8aM1j4X0$>>
>
> Looking forward to the results of the discussion on Monday! I won't be
> able to join the telco discussion, but I'm available via email for any
> questions or clarifications that arise.
>
>
> On Sat, Oct 18, 2025 at 1:09 PM Harald Oehlmann
> <har...@el...<mailto:har...@el...> <mailto:har...@el...<mailto:har...@el...>>> wrote:
>
> Am 17.10.2025 um 23:22 schrieb EricT:
> > Hello Tcl Core Team,
> >
> > I have developed a working prototype implementation of TIP 672,
> which
> > adds the $(expression) syntax as a more intuitive alternative to
> [expr
> > {expression}].
> >
> > Repository: https://github.com/rocketship88/tcl-tip-672-prototype [github.com]<https://urldefense.com/v3/__https://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8buJgHl4$>
> <https://github.com/rocketship88/tcl-tip-672-prototype [github.com]<https://urldefense.com/v3/__https://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8buJgHl4$>>
> > <https://github.com/rocketship88/tcl-tip-672-prototype [github.com]<https://urldefense.com/v3/__https://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8buJgHl4$> <https://
> github.com/rocketship88/tcl-tip-672-prototype [github.com]<https://urldefense.com/v3/__http://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8aM1j4X0$>>>
> >
> > The implementation is minimal, modifying only two files
> (tclParse.c and
> > tclNamesp.c) with approximately 100 lines of changes. The key
> > modification converts the existing two-way branch in
> Tcl_ParseVarName to
> > a three-way branch, with the new path handling $(...) by creating a
> > synthetic [expr {...}] command string.
> >
> > Key Accomplishments:
> >
> > Full bytecode compilation: The synthetic string approach integrates
> > seamlessly with the existing compiler, producing identical optimized
> > bytecode as [expr {...}]. The disassembler output (shown in the
> README)
> > demonstrates efficient variable loading with no runtime parsing
> overhead.
> >
> > Proven approach: Jim Tcl has used this syntax successfully for years
> >
> > Comprehensive testing: Works correctly with string interpolation,
> > variable scoping, error handling, and interactive mode
> >
> > Known Limitations:
> >
> > Memory leak: The synthetic string is allocated but not tracked for
> > cleanup in Tcl_FreeParse. This requires core team guidance on the
> > preferred solution (modify Tcl_Parse structure vs. thread-local
> tracking).
> >
> > Error messages: Currently show the synthetic command rather than the
> > original $(...) syntax, though this is arguably helpful for
> debugging.
> >
> > Questions for the Team:
> >
> > What is the preferred approach for tracking synthetic strings for
> cleanup?
> > Is this prototype architecture acceptable for Tcl 9.x?
> > Are there concerns with the synthetic string approach that I
> should address?
> >
> > The complete implementation with side-by-side diffs is available
> in the
> > repository. I'm happy to refine the code based on your feedback and
> > would appreciate any guidance on moving this forward.
> >
> > Best regards,
> > Eric
> Eric,
> great proposal, thank you !
>
> Perhaps, we may discuss this on Monday in the biweekly telco.
> I am also excited and in favor to this.
> Nevertheless, "$(a)" is to my knowledge a quite common syntax for
> arrays. We have already killed less disruptive proposals (see optional
> "- args") by endless discussions and getting nowhere.
> Hope, this will be fruitful.
>
> In addition, I would like to add the Tk test reform by the other
> Eric to
> the biweekly topics.
> Here is a revised agenda proposal proposal:
>
> Top 1) Release calender (TIP 713)
> - 9.0.3: October (2 weeks left)
> - 9.1a1: November (6 weeks left)
> Top 2) TIP 734 nested mutex (go or not)
> Top 3) TIP 733: accessability (test status)
> Top 4) TIP 732: TCL library path (discussion)
> Top 5) TIP 731: use C enums (no brainer?)
> Top 6) TIP 720: new bytecodes (final, great! Any issues?)
> Top 7) TIP 721: Tcl_AttemptGetString
> Top 8) TIP 715: supported build systems
> Top 9) $($a+$b) syntax for expressions
> Top 10) Tk test reform by Eric (Thanks !)
> Top 11) Tcl depot and awthemes ?
> Top 12) AOB
> Top 13) Next meeting:
> 3rd of November 12:00 UTC.
> Daytime saving time ends 2nd of November in US, 26th of
> October in
> Europe.
> Will we keep 12:00 UTC ? Or 13:00 UTC, so Don has 8:00 AM?
>
> Take care,
> Harald
_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core [lists.sourceforge.net]<https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/tcl-core__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl881_NXzE$>
_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core [lists.sourceforge.net]<https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/tcl-core__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl881_NXzE$>
_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core [lists.sourceforge.net]<https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/tcl-core__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl881_NXzE$>
_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core [lists.sourceforge.net]<https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/tcl-core__;!!PDiH4ENfjr2_Jw!Eot1BTBAjUrPeBxEg8km_8tqRFXpHzfHkiNZX7qagLMYARtCjwfj3_uX-YkRBZbGR4sbbT_P-u3u9zqo38ySlCQMhbQ$>
|
