Re: [TCLCORE] Prototype Implementation of TIP 672 - $(expr) Syntax

[Donal F. <don...@ma...>]

​
EricT
2025-10-21 04:12
1. Shell precedent: The $(command) syntax is well-established in bash/sh for command substitution. Tcl users familiar with shell scripting would find this natural, not confusing.
​
Surely the better shell precedent would be the $((expression)) form? That would also be quite a bit less likely to clash with existing use cases. Hmm, a quick search with https://github.com/search?q=%24%28%28+language%3ATcl&type=code gives 29 hits (most of which appear to be false positives; the real hits seem to be from one place in JimTcl itself), as opposed to 4k hits when looking for $( in Tcl code; two orders of magnitude less usage. One of the really nice things about a resource like GitHub is that one can quickly search for language syntax features across a lot of code and get a good feeling for practical compatibility.

While $(([llength $x] - 17)) is no shorter than {=}{[llength $x] - 17}, it's definitely easier to type!

(Eric: Don't lose heart! These discussions are a lot less acrimonious than the discussion over what became {*} was.)

Donal.

________________________________
From: EricT <tw...@gm...>
Sent: Tuesday, October 21, 2025 04:11
To: Brian Griffin <bri...@ea...>
Cc: tc...@ro... <tc...@ro...>; Tcl Core List <tcl...@li...>
Subject: Re: [TCLCORE] Prototype Implementation of TIP 672 - $(expr) Syntax

Thanks for the feedback, Brian! I understand your concern about the semantic distinction between $ (value) and [ ] (execution).

I'd offer a few thoughts:

1. Shell precedent: The $(command) syntax is well-established in bash/sh for command substitution. Tcl users familiar with shell scripting would find this natural, not confusing.

2. $ already involves execution: Even $var involves command execution internally (TclGetVar), it's just optimized. The distinction between "value substitution" and "command execution" is somewhat artificial at the implementation level.

3.  expr is special and has historical precedent: Unlike arbitrary commands, expr is used so frequently that syntactic sugar makes sense - similar to how $var is sugar for [set var]. We optimize the common case. I believe there was a time in early Tcl before $var existed when [set var] was the only choice - if so, adding $var as syntactic sugar for variable substitution was a usability win. $(expr) follows the same pattern - sugar for the extremely common [expr {...}] case.

4. Consistency: $(expr) fits the pattern of "$ means substitute something here" - whether a variable, array element, or expression result. Modern languages like Python have embraced similar concepts with f-strings that allow {expression} for inline evaluation. $(expr) brings this convenience to Tcl while maintaining our substitution semantics.

That said, I appreciate the philosophical consistency argument. Do you see the security benefits (auto-bracing) as compelling enough to outweigh the semantic concern?

Did you see the email I sent you about lseq? Maybe it landed in your spam folder - that happened once before as I recall.

Eric


On Mon, Oct 20, 2025 at 7:19 PM Brian Griffin <bri...@ea...<mailto:bri...@ea...>> wrote:
I like the idea of simplifying expression substitutions.  The problem I have is that the '$' is defined to be a value substitution. Running a command (execution) is a [] substitution, not a '$' substitution.  Conflating the two modes of substitution can cause confusion for newbies, and even experienced programmers.

I recognize that this view is a very subtle distinction, but I think it's important.

Think of it like conflating * vs & in C.  (Maybe not that bad, but similar)

-Brian

On Oct 20, 2025, at 07:22, EricT <tw...@gm...<mailto:tw...@gm...>> wrote:

Regarding the discussion in the telco today. If the incompatibility discussed is the known empty array issue, then for that I would propose the following:

There could be a pragma or tcl_dollar_expr global variable with several settings.

1. Work in compatibility mode, i.e. as though the feature was not implemented at all.
2. Work in diagnostic mode, throw an error for any $(...) use
3. Work in full implementation mode

The code to do this, given a global C variable to check would be trivial.

This value could default to 1. for at least 9.1 or 9.2 but there could be an option, similar to tclsh -encoding, that would allow the pragma variable to be set before any init script code is run. This would give developers the opportunity to set the variable to mode 2 in order to easily track down uses of $(index) before any code were to begin using $(expr).

I think it would be quite rare if this syntax change would affect binary extensions, so it should be only with script level coding.

The fix to any code that is using the empty array variable, is to change from,

    set val $(index)
    set val $($var)  - or any other valid index string

to

   set val ${(index)}
   set val ${($var)}

Likewise for any other use of the $(...) substitution in current use.

This change can begin to be made right away, since it is a compatible syntax that does not change the semantics.  Once all the init time script code or packages that use the un-braced syntax are found and changed, the feature could begin to be used by those who are interested.

On the other hand, if the incompatibility is something other than this empty array, I'd be most appreciative to know what it is.

thanks

Eric


On Sun, Oct 19, 2025 at 3:44 AM EricT <tw...@gm...<mailto:tw...@gm...>> wrote:
Harald,

Thank you for your willingness to sponsor!

To clarify: I am not the author of Jim Tcl - I'm just a long-time Tcl user who, in retirement, wanted to give something back to the Tcl community. Jim Tcl's successful use of this syntax for years demonstrates that the concept is viable.

Regarding the telco: I attempted to join previously but was unsuccessful with the setup. I won't be able to participate on Monday, but I'm available via email for any questions or clarifications that arise from the discussion.

I don't currently have write access to either the Tcl Fossil repository or the TIP repository, so I'm unable to make changes directly. At nearly 80 years old, I'm not comfortable learning new version control systems on my own. If the TIP moves forward, I'd need guidance and assistance with the Fossil workflow, or perhaps someone from the core team who shares our interest in this TIP could handle the integration based on my GitHub prototype.

I'm currently developing a comprehensive test suite in standard tcltest format, with particular focus on edge cases.

Looking forward to hearing how Monday's discussion goes!

Best regards,
Eric


On Sun, Oct 19, 2025 at 2:05 AM Harald Oehlmann <har...@el...<mailto:har...@el...>> wrote:
Eric,
sponsoring is no problem. The point is to get positive votes.
This requires a positive discussion.
You are the Author of Jim?
Could you participate on the telco on Monday?

Do you have write access to the tcl fossil? We would need the
implementation there.

Thanks for all,
Harald

Am 18.10.2025 um 22:58 schrieb EricT:
> Thank you for the positive feedback and for raising this in Monday's
> telco! I'm encouraged by your support.
>
> Regarding $(a) - you're right that reading an empty array element with a
> variable index is a valid construct. However, this is explicitly
> addressed in the TIP and the repository README. The workarounds are
> straightforward:
>
>    ${(a)}        # Braced form
>    [set (a)]     # Command substitution
>
> Both still work. In fact, code using $(varname) could be proactively
> modified to use ${(varname)} to indicate the clear intent of an empty
> array reference, which improves readability. The security, performance,
> and usability benefits of $(...) seemed to justify this trade-off for
> Tcl 9.x where some incompatibilities are expected.
>
> Given your interest in the feature, would you be willing to consider
> sponsoring TIP 672? The implementation is working and minimal (~100
> lines across two files), with the main open question being the preferred
> approach for tracking synthetic strings for cleanup. Your guidance on
> that architectural decision would be particularly valuable.
>
> The prototype repository with full implementation and examples is here:
>
> https://github.com/rocketship88/tcl-tip-672-prototype [github.com]<https://urldefense.com/v3/__https://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8buJgHl4$> <https://
> github.com/rocketship88/tcl-tip-672-prototype [github.com]<https://urldefense.com/v3/__http://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8aM1j4X0$>>
>
> Looking forward to the results of the discussion on Monday! I won't be
> able to join the telco discussion, but I'm available via email for any
> questions or clarifications that arise.
>
>
> On Sat, Oct 18, 2025 at 1:09 PM Harald Oehlmann
> <har...@el...<mailto:har...@el...> <mailto:har...@el...<mailto:har...@el...>>> wrote:
>
>     Am 17.10.2025 um 23:22 schrieb EricT:
>      > Hello Tcl Core Team,
>      >
>      > I have developed a working prototype implementation of TIP 672,
>     which
>      > adds the $(expression) syntax as a more intuitive alternative to
>     [expr
>      > {expression}].
>      >
>      > Repository: https://github.com/rocketship88/tcl-tip-672-prototype [github.com]<https://urldefense.com/v3/__https://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8buJgHl4$>
>     <https://github.com/rocketship88/tcl-tip-672-prototype [github.com]<https://urldefense.com/v3/__https://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8buJgHl4$>>
>      > <https://github.com/rocketship88/tcl-tip-672-prototype [github.com]<https://urldefense.com/v3/__https://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8buJgHl4$> <https://
>     github.com/rocketship88/tcl-tip-672-prototype [github.com]<https://urldefense.com/v3/__http://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8aM1j4X0$>>>
>      >
>      > The implementation is minimal, modifying only two files
>     (tclParse.c and
>      > tclNamesp.c) with approximately 100 lines of changes. The key
>      > modification converts the existing two-way branch in
>     Tcl_ParseVarName to
>      > a three-way branch, with the new path handling $(...) by creating a
>      > synthetic [expr {...}] command string.
>      >
>      > Key Accomplishments:
>      >
>      > Full bytecode compilation: The synthetic string approach integrates
>      > seamlessly with the existing compiler, producing identical optimized
>      > bytecode as [expr {...}]. The disassembler output (shown in the
>     README)
>      > demonstrates efficient variable loading with no runtime parsing
>     overhead.
>      >
>      > Proven approach: Jim Tcl has used this syntax successfully for years
>      >
>      > Comprehensive testing: Works correctly with string interpolation,
>      > variable scoping, error handling, and interactive mode
>      >
>      > Known Limitations:
>      >
>      > Memory leak: The synthetic string is allocated but not tracked for
>      > cleanup in Tcl_FreeParse. This requires core team guidance on the
>      > preferred solution (modify Tcl_Parse structure vs. thread-local
>     tracking).
>      >
>      > Error messages: Currently show the synthetic command rather than the
>      > original $(...) syntax, though this is arguably helpful for
>     debugging.
>      >
>      > Questions for the Team:
>      >
>      > What is the preferred approach for tracking synthetic strings for
>     cleanup?
>      > Is this prototype architecture acceptable for Tcl 9.x?
>      > Are there concerns with the synthetic string approach that I
>     should address?
>      >
>      > The complete implementation with side-by-side diffs is available
>     in the
>      > repository. I'm happy to refine the code based on your feedback and
>      > would appreciate any guidance on moving this forward.
>      >
>      > Best regards,
>      > Eric
>     Eric,
>     great proposal, thank you !
>
>     Perhaps, we may discuss this on Monday in the biweekly telco.
>     I am also excited and in favor to this.
>     Nevertheless, "$(a)" is to my knowledge a quite common syntax for
>     arrays. We have already killed less disruptive proposals (see optional
>     "- args") by endless discussions and getting nowhere.
>     Hope, this will be fruitful.
>
>     In addition, I would like to add the Tk test reform by the other
>     Eric to
>     the biweekly topics.
>     Here is a revised agenda proposal proposal:
>
>     Top 1) Release calender (TIP 713)
>     - 9.0.3: October (2 weeks left)
>     - 9.1a1: November (6 weeks left)
>     Top 2) TIP 734 nested mutex (go or not)
>     Top 3) TIP 733: accessability (test status)
>     Top 4) TIP 732: TCL library path (discussion)
>     Top 5) TIP 731: use C enums (no brainer?)
>     Top 6) TIP 720: new bytecodes (final, great! Any issues?)
>     Top 7) TIP 721: Tcl_AttemptGetString
>     Top 8) TIP 715: supported build systems
>     Top 9) $($a+$b) syntax for expressions
>     Top 10) Tk test reform by Eric (Thanks !)
>     Top 11) Tcl depot and awthemes ?
>     Top 12) AOB
>     Top 13) Next meeting:
>           3rd of November 12:00 UTC.
>           Daytime saving time ends 2nd of November in US, 26th of
>     October in
>     Europe.
>           Will we keep 12:00 UTC ? Or 13:00 UTC, so Don has 8:00 AM?
>
>     Take care,
>     Harald
_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core [lists.sourceforge.net]<https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/tcl-core__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl881_NXzE$>
_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core [lists.sourceforge.net]<https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/tcl-core__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl881_NXzE$>

_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core [lists.sourceforge.net]<https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/tcl-core__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl881_NXzE$>