Re: [TCLCORE] Prototype Implementation of TIP 672 - $(expr) Syntax

[EricT <tw...@gm...>]

Thanks Arjen for the heads-up! This is exactly why I think a phased
rollout with the pragma approach is the best path forward - it gives
time to identify and update all such uses to the ${(index)} syntax
before enabling the feature.

Colin:

The "macro" approach is something I think could have other uses as
well. In the case of $(expr) it's strength is in the security aspect,
which generates fully braced expr.

By having an easier way to write expressions, users would likely
gravitate to it over time thus improving overall Tcl security. Using
{=}{...} instead of [expr {...}] doesn't seem as compelling for those
seeking cleaner syntax who would also get better security as a
by-product.

I hesitate to mention this here, but I've found a new injection
exploit in 9.x that needs to be fixed. The fix would be much less
traumatic if $(expr) were implemented in 9.x, thus I feel that waiting
for version 10, which could be many years away, is a mistake. I think
a pragma (or knob) approach would be a better plan.

I also see the {=}{expr} approach as a much more difficult
implementation challenge.

Firstly, {*} has to begin a word, it cannot be used inside of a
bareword or a quoted word. That makes it much easier to deal with. For
example, you can't have {*}{1 2 3}xyz or you get an error of extra
characters after close-brace. But you can have - set foo $(1+2)xyz -
and end up with 3xyz.

The parser has separate mechanisms for scanning quoted strings and
bareword strings. The $ trigger works in both of those cases, and need
not begin at a word boundary. I think that trying to wedge {=} into
the parser could likely result in many more edge cases. Certainly, it
would take more than the 100 lines of code that $(expr) required.

And I suspect somewhere in the wild there's a quoted string that has
that pattern in it, perhaps {=}{=}{=} which would be enough to trigger
the substitution. Users all know that if they use a $ in a quoted
string, that it's special.

There is one other edge case with $(expr): the subst command currently
treats it as variable substitution. This means:
subst -nocommands {$(expr)}    # Doesn't work - still substitutessubst
-novariables {$(expr)}   # Works - suppresses substitution

This is addressable but needs consideration.

However, I have not fully investigated how {=}{expr} would interact
with subst, so it may have similar challenges.

--Eric


On Mon, Oct 20, 2025 at 1:46 PM Arjen Markus via Tcl-Core <
tcl...@li...> wrote:

> Hi Eric, everyone,
>
>
>
> Just for your information: The one extension that I know of that uses an
> anonymous array is stooop – Simple Tcl-only object-oriented programming –
> by Jean-Luc Fontaine. It is part of Tcllib.
>
>
>
> Regards,
>
>
>
> Arjen
>
>
>
> *From:* EricT <tw...@gm...>
> *Sent:* maandag 20 oktober 2025 19:45
> *To:* Colin Macleod <col...@ya...>
> *Cc:* tcl...@li...
> *Subject:* Re: [TCLCORE] Prototype Implementation of TIP 672 - $(expr)
> Syntax
>
>
>
> *Caution:* This message was sent from outside of Deltares. Please do not
> click links or open attachments unless you recognize the source of this
> email and know the content is safe. Please report all suspicious emails to "
> Ser...@de..." as an attachment.
>
>
>
> Besides for the jimtcl compatibility and its track record in use, there's
> another reason I like the $(expr) notation and that is that it acts in all
> ways like $var, so that what people have learned, such as how to use $
> substitution in event callbacks, either by [list puts $var] or {puts $var}
> depending on whether or not they want delayed evaluation, would be similar
> for [list puts $(expr)] and {puts $(expr)}.
>
>
>
> One other issue that came up in my testing was that my first upload to
> github a few days ago required that ()'s be balanced in the entire
> expression, including when there were command callbacks, such as
>
>
>
> [string length abc(def]+$n
>
>
>
> That would have needed to be escaped as abc\(def, but that restriction has
> been eliminated in the most recent code in my repository. It turns out that
> ()'s need to be balanced ONLY in the portions of an expression that are
> arithmetical, where, for example,  (((1+2))) balancing would be required.
> The fix is in the latest upload to the repository. This makes it compatible
> with expr which also does not require balancing of ()'s in command callback
> strings whether bare or quoted.
>
>
>
> One other fix was to eliminate the code that was changing newlines to
> spaces when found in an expression, since that was thought to be a problem,
> but turns out not to be the case. expr has no problem with newline as
> whitespace.
>
>
>
> --Eric
>
>
>
>
>
>
>
>
>
> On Mon, Oct 20, 2025 at 9:31 AM Colin Macleod via Tcl-Core <
> tcl...@li...> wrote:
>
> I would say {=}{expression} is more Tcl-ish, though two characters
> longer.  It follows the precedent of {*}, and avoids the incompatibility of
> $(expression).  I wonder if that could also be implemented by a simple
> macro-type substitution at the parsing stage, as Eric has proposed.  If {=}
> was TIP-ed I would happily withdraw my competing TIP 676.
>
> Colin.
>
> On 20/10/2025 16:11, da Silva, Peter J wrote:
>
> Obviously I think this is a great idea and I think anyone who actually
> uses $(thing) in production code is crazy, but when I brought it up it got
> shot down because it was apparently found in the wild in production code.
>
>
>
> Pity it didn’t get into 9.0 :) but this would be a good era to do it in.
>
>
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>
> DISCLAIMER: This message is intended exclusively for the addressee(s) and
> may contain confidential and privileged information. If you are not the
> intended recipient please notify the sender immediately and destroy this
> message. Unauthorized use, disclosure or copying of this message is
> strictly prohibited. The foundation 'Stichting Deltares', which has its
> seat at Delft, The Netherlands, Commercial Registration Number 41146461, is
> not liable in any way whatsoever for consequences and/or damages resulting
> from the improper, incomplete and untimely dispatch, receipt and/or content
> of this e-mail.
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>