Menu
▾
▴
Re: [TCLCORE] Prototype Implementation of TIP 672 - $(expr) Syntax
|
From: EricT <tw...@gm...> - 2025-10-21 12:36:25
|
Hi Donal:
Yes, I should have been more careful with that. I see that it is actually
$((...)) in bash.
I think the best approach is to find all the uses by both searching and
also a pragma that optionally will cause any use of $() to be flagged with
an error. Then convert to using one of the two alternative methods.
I understand empty array names exist in production code. It's also
important to note that ${()} doesn't work for all cases - it only handles
literal indices. When the index involves variable substitution (like
$($var)), users must use [set (...)] instead.
The migration path is clear I believe: use [set (...)] which works for all
cases, both before and after the feature is enabled.
Thanks again for the feedback.
Eric
On Tue, Oct 21, 2025 at 1:55 AM Donal Fellows <
don...@ma...> wrote:
>
> EricT
> 2025-10-21 04:12
> 1. Shell precedent: The $(command) syntax is well-established in bash/sh
> for command substitution. Tcl users familiar with shell scripting would
> find this natural, not confusing.
>
> Surely the better shell precedent would be the $((expression)) form? That
> would also be quite a bit less likely to clash with existing use cases. Hmm,
> a quick search with
> https://github.com/search?q=%24%28%28+language%3ATcl&type=code gives 29
> hits (most of which appear to be false positives; the real hits seem to be
> from one place in JimTcl itself), as opposed to 4k hits when looking for $(
> in Tcl code; two orders of magnitude less usage. One of the *really* nice
> things about a resource like GitHub is that one can *quickly* search for
> language syntax features across a lot of code and get a good feeling for
> practical compatibility.
>
> While $(([llength $x] - 17)) is no shorter than {=}{[llength $x] - 17},
> it's definitely easier to type!
>
> (Eric: Don't lose heart! These discussions are a *lot* less acrimonious
> than the discussion over what became {*} was.)
>
> Donal.
>
> ------------------------------
> *From:* EricT <tw...@gm...>
> *Sent:* Tuesday, October 21, 2025 04:11
> *To:* Brian Griffin <bri...@ea...>
> *Cc:* tc...@ro... <tc...@ro...>; Tcl Core List <
> tcl...@li...>
> *Subject:* Re: [TCLCORE] Prototype Implementation of TIP 672 - $(expr)
> Syntax
>
> Thanks for the feedback, Brian! I understand your concern about the
> semantic distinction between $ (value) and [ ] (execution).
>
> I'd offer a few thoughts:
>
> 1. Shell precedent: The $(command) syntax is well-established in bash/sh
> for command substitution. Tcl users familiar with shell scripting would
> find this natural, not confusing.
>
> 2. $ already involves execution: Even $var involves command execution
> internally (TclGetVar), it's just optimized. The distinction between "value
> substitution" and "command execution" is somewhat artificial at the
> implementation level.
>
> 3. expr is special and has historical precedent: Unlike arbitrary
> commands, expr is used so frequently that syntactic sugar makes sense -
> similar to how $var is sugar for [set var]. We optimize the common case. I
> believe there was a time in early Tcl before $var existed when [set var]
> was the only choice - if so, adding $var as syntactic sugar for variable
> substitution was a usability win. $(expr) follows the same pattern - sugar
> for the extremely common [expr {...}] case.
>
> 4. Consistency: $(expr) fits the pattern of "$ means substitute something
> here" - whether a variable, array element, or expression result. Modern
> languages like Python have embraced similar concepts with f-strings that
> allow {expression} for inline evaluation. $(expr) brings this convenience
> to Tcl while maintaining our substitution semantics.
>
> That said, I appreciate the philosophical consistency argument. Do you see
> the security benefits (auto-bracing) as compelling enough to outweigh the
> semantic concern?
>
> Did you see the email I sent you about lseq? Maybe it landed in your spam
> folder - that happened once before as I recall.
>
> Eric
>
>
> On Mon, Oct 20, 2025 at 7:19 PM Brian Griffin <bri...@ea...>
> wrote:
>
> I like the idea of simplifying expression substitutions. The problem I
> have is that the '$' is defined to be a value substitution. Running a
> command (execution) is a [] substitution, not a '$' substitution.
> Conflating the two modes of substitution can cause confusion for newbies,
> and even experienced programmers.
>
> I recognize that this view is a very subtle distinction, but I think it's
> important.
>
> Think of it like conflating * vs & in C. (Maybe not that bad, but similar)
>
> -Brian
>
> On Oct 20, 2025, at 07:22, EricT <tw...@gm...> wrote:
>
> Regarding the discussion in the telco today. If the incompatibility
> discussed is the known empty array issue, then for that I would propose the
> following:
>
> There could be a pragma or tcl_dollar_expr global variable with several
> settings.
>
> 1. Work in compatibility mode, i.e. as though the feature was not
> implemented at all.
> 2. Work in diagnostic mode, throw an error for any $(...) use
> 3. Work in full implementation mode
>
> The code to do this, given a global C variable to check would be trivial.
>
> This value could default to 1. for at least 9.1 or 9.2 but there could be
> an option, similar to tclsh -encoding, that would allow the pragma variable
> to be set before any init script code is run. This would give developers
> the opportunity to set the variable to mode 2 in order to easily track down
> uses of $(index) before any code were to begin using $(expr).
>
> I think it would be quite rare if this syntax change would affect binary
> extensions, so it should be only with script level coding.
>
> The fix to any code that is using the empty array variable, is to change
> from,
>
> set val $(index)
> set val $($var) - or any other valid index string
>
> to
>
> set val ${(index)}
> set val ${($var)}
>
> Likewise for any other use of the $(...) substitution in current use.
>
> This change can begin to be made right away, since it is a
> compatible syntax that does not change the semantics. Once all the init
> time script code or packages that use the un-braced syntax are found and
> changed, the feature could begin to be used by those who are interested.
>
> On the other hand, if the incompatibility is something other than this
> empty array, I'd be most appreciative to know what it is.
>
> thanks
>
> Eric
>
>
> On Sun, Oct 19, 2025 at 3:44 AM EricT <tw...@gm...> wrote:
>
> Harald,
>
> Thank you for your willingness to sponsor!
>
> To clarify: I am not the author of Jim Tcl - I'm just a long-time Tcl user
> who, in retirement, wanted to give something back to the Tcl community. Jim
> Tcl's successful use of this syntax for years demonstrates that the concept
> is viable.
>
> Regarding the telco: I attempted to join previously but was unsuccessful
> with the setup. I won't be able to participate on Monday, but I'm available
> via email for any questions or clarifications that arise from the
> discussion.
>
> I don't currently have write access to either the Tcl Fossil repository or
> the TIP repository, so I'm unable to make changes directly. At nearly 80
> years old, I'm not comfortable learning new version control systems on my
> own. If the TIP moves forward, I'd need guidance and assistance with the
> Fossil workflow, or perhaps someone from the core team who shares our
> interest in this TIP could handle the integration based on my GitHub
> prototype.
>
> I'm currently developing a comprehensive test suite in standard tcltest
> format, with particular focus on edge cases.
>
> Looking forward to hearing how Monday's discussion goes!
>
> Best regards,
> Eric
>
>
> On Sun, Oct 19, 2025 at 2:05 AM Harald Oehlmann <
> har...@el...> wrote:
>
> Eric,
> sponsoring is no problem. The point is to get positive votes.
> This requires a positive discussion.
> You are the Author of Jim?
> Could you participate on the telco on Monday?
>
> Do you have write access to the tcl fossil? We would need the
> implementation there.
>
> Thanks for all,
> Harald
>
> Am 18.10.2025 um 22:58 schrieb EricT:
> > Thank you for the positive feedback and for raising this in Monday's
> > telco! I'm encouraged by your support.
> >
> > Regarding $(a) - you're right that reading an empty array element with a
> > variable index is a valid construct. However, this is explicitly
> > addressed in the TIP and the repository README. The workarounds are
> > straightforward:
> >
> > ${(a)} # Braced form
> > [set (a)] # Command substitution
> >
> > Both still work. In fact, code using $(varname) could be proactively
> > modified to use ${(varname)} to indicate the clear intent of an empty
> > array reference, which improves readability. The security, performance,
> > and usability benefits of $(...) seemed to justify this trade-off for
> > Tcl 9.x where some incompatibilities are expected.
> >
> > Given your interest in the feature, would you be willing to consider
> > sponsoring TIP 672? The implementation is working and minimal (~100
> > lines across two files), with the main open question being the preferred
> > approach for tracking synthetic strings for cleanup. Your guidance on
> > that architectural decision would be particularly valuable.
> >
> > The prototype repository with full implementation and examples is here:
> >
> > https://github.com/rocketship88/tcl-tip-672-prototype [github.com]
> <https://urldefense.com/v3/__https://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8buJgHl4$>
> <https://
> > github.com/rocketship88/tcl-tip-672-prototype [github.com]
> <https://urldefense.com/v3/__http://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8aM1j4X0$>
> >
> >
> > Looking forward to the results of the discussion on Monday! I won't be
> > able to join the telco discussion, but I'm available via email for any
> > questions or clarifications that arise.
> >
> >
> > On Sat, Oct 18, 2025 at 1:09 PM Harald Oehlmann
> > <har...@el... <mailto:har...@el...>>
> wrote:
> >
> > Am 17.10.2025 um 23:22 schrieb EricT:
> > > Hello Tcl Core Team,
> > >
> > > I have developed a working prototype implementation of TIP 672,
> > which
> > > adds the $(expression) syntax as a more intuitive alternative to
> > [expr
> > > {expression}].
> > >
> > > Repository: https://github.com/rocketship88/tcl-tip-672-prototype
> [github.com]
> <https://urldefense.com/v3/__https://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8buJgHl4$>
> > <https://github.com/rocketship88/tcl-tip-672-prototype [github.com]
> <https://urldefense.com/v3/__https://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8buJgHl4$>
> >
> > > <https://github.com/rocketship88/tcl-tip-672-prototype
> [github.com]
> <https://urldefense.com/v3/__https://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8buJgHl4$>
> <https://
> > github.com/rocketship88/tcl-tip-672-prototype [github.com]
> <https://urldefense.com/v3/__http://github.com/rocketship88/tcl-tip-672-prototype__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl8aM1j4X0$>
> >>
> > >
> > > The implementation is minimal, modifying only two files
> > (tclParse.c and
> > > tclNamesp.c) with approximately 100 lines of changes. The key
> > > modification converts the existing two-way branch in
> > Tcl_ParseVarName to
> > > a three-way branch, with the new path handling $(...) by creating
> a
> > > synthetic [expr {...}] command string.
> > >
> > > Key Accomplishments:
> > >
> > > Full bytecode compilation: The synthetic string approach
> integrates
> > > seamlessly with the existing compiler, producing identical
> optimized
> > > bytecode as [expr {...}]. The disassembler output (shown in the
> > README)
> > > demonstrates efficient variable loading with no runtime parsing
> > overhead.
> > >
> > > Proven approach: Jim Tcl has used this syntax successfully for
> years
> > >
> > > Comprehensive testing: Works correctly with string interpolation,
> > > variable scoping, error handling, and interactive mode
> > >
> > > Known Limitations:
> > >
> > > Memory leak: The synthetic string is allocated but not tracked for
> > > cleanup in Tcl_FreeParse. This requires core team guidance on the
> > > preferred solution (modify Tcl_Parse structure vs. thread-local
> > tracking).
> > >
> > > Error messages: Currently show the synthetic command rather than
> the
> > > original $(...) syntax, though this is arguably helpful for
> > debugging.
> > >
> > > Questions for the Team:
> > >
> > > What is the preferred approach for tracking synthetic strings for
> > cleanup?
> > > Is this prototype architecture acceptable for Tcl 9.x?
> > > Are there concerns with the synthetic string approach that I
> > should address?
> > >
> > > The complete implementation with side-by-side diffs is available
> > in the
> > > repository. I'm happy to refine the code based on your feedback
> and
> > > would appreciate any guidance on moving this forward.
> > >
> > > Best regards,
> > > Eric
> > Eric,
> > great proposal, thank you !
> >
> > Perhaps, we may discuss this on Monday in the biweekly telco.
> > I am also excited and in favor to this.
> > Nevertheless, "$(a)" is to my knowledge a quite common syntax for
> > arrays. We have already killed less disruptive proposals (see
> optional
> > "- args") by endless discussions and getting nowhere.
> > Hope, this will be fruitful.
> >
> > In addition, I would like to add the Tk test reform by the other
> > Eric to
> > the biweekly topics.
> > Here is a revised agenda proposal proposal:
> >
> > Top 1) Release calender (TIP 713)
> > - 9.0.3: October (2 weeks left)
> > - 9.1a1: November (6 weeks left)
> > Top 2) TIP 734 nested mutex (go or not)
> > Top 3) TIP 733: accessability (test status)
> > Top 4) TIP 732: TCL library path (discussion)
> > Top 5) TIP 731: use C enums (no brainer?)
> > Top 6) TIP 720: new bytecodes (final, great! Any issues?)
> > Top 7) TIP 721: Tcl_AttemptGetString
> > Top 8) TIP 715: supported build systems
> > Top 9) $($a+$b) syntax for expressions
> > Top 10) Tk test reform by Eric (Thanks !)
> > Top 11) Tcl depot and awthemes ?
> > Top 12) AOB
> > Top 13) Next meeting:
> > 3rd of November 12:00 UTC.
> > Daytime saving time ends 2nd of November in US, 26th of
> > October in
> > Europe.
> > Will we keep 12:00 UTC ? Or 13:00 UTC, so Don has 8:00 AM?
> >
> > Take care,
> > Harald
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
> [lists.sourceforge.net]
> <https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/tcl-core__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl881_NXzE$>
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
> [lists.sourceforge.net]
> <https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/tcl-core__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl881_NXzE$>
>
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
> [lists.sourceforge.net]
> <https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/tcl-core__;!!PDiH4ENfjr2_Jw!Fnw2VZOkJIt0hYAyi7uMJpHn2oP79DlZ5A8fqXORtGo9BUUJ35XItI-pNGmBahGZVIO3H-Kc6D13fiFBnEl881_NXzE$>
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>
|
