Re: [TCLCORE] Fwd: TIP 672 Implementation Complete - Ready for Sponsorship

[Donal F. <don...@ma...>]

​
Conclusion : There is no real choice
​
That's an incorrect conclusion. At least one of the priors you use in reaching it is invalid.

The core of the proposal is that the sequence $( start a new syntax category (with what follows being an expression to the point where there's a matching parenthesis using an appropriate recursive matching algorithm, with a similar model to how command substitutions work). This new syntactic entity will be called (almost certainly) an expression substitution. That it starts with $ is good; existing code that inserts backslashes before dollar signs to make things subst-safe will not be surprised; normally we'd recommend such approaches instead now switch to using regsub -command, due to the way subst works, but that's a longer-term change-over.

That the C code inside the Tcl parsing engine will need to change to accommodate a new syntax category isn't a deep insight. Of course it will need to change. Such change is what is being proposed!

Right now, the real discussion is what the sequence of literal characters in a Tcl script to introduce the new syntax should be. The options in play are:


  1.
Do nothing. Stick with [expr {expression...}]. This option is always open to us.
  2.
Use $(expression...)
  3.
Use $((expression...))
  4.
Use [={expression...}]
  5.
Use [= expression...]
  6.
Use [(expression...)]
  7.
Use something else (I've probably missed an option or two from this painting of the bikeshed)

Of the options above, 2 has real compatibility issues (i.e., we know it will clash with existing code in Tcllib) and 5 probably has semantic issues (because it was proposed as an actual command; anything that introduces uncontrolled double substitution is a no-no at this point). Option 3 has been assessed for practical compatibility, and should be fine (except for a few places in JimTcl, written by someone already in this conversation). Option 4 manages to be ugly, but is probably OK in terms of amount of in-the-wild usage. Option 6 hasn't been assessed yet.

For all options that introduce a new syntactic substitution class, a change to subst is needed to work with it (Jan's outlined this adequately) and there should be consideration whether the other mini-languages inside Tcl should change (we have several).

Donal.

________________________________
From: Florent Merlet <flo...@gm...>
Sent: Thursday, October 23, 2025 09:32
To: tcl...@li... <tcl...@li...>
Subject: Re: [TCLCORE] Fwd: TIP 672 Implementation Complete - Ready for Sponsorship

Hello Jan, So we can admit that the implementation is not ready. In fact, expression is modelised like a variable and it can't nest. Fondamentally, the problem is that the « $ » sign allways has been related to the concept of variable, whereas


Hello Jan,

So we can admit that the implementation is not ready.

In fact, expression is modelised like a variable and it can't nest.

Fondamentally, the problem is that the « $ » sign allways has been related to the concept of variable, whereas an expression is related to the concept of command.

« variables » have caracteristics that differ from the one of « commands » .

« commands » can nest, whereas « variables » don't.

These distinct caracteristics were translated directly in the programm flow, because, at a certain degree, there must be an analogy between the flow of the programm and the caracteristics of the concept.

In Parse_token, the « $ » branch doesn't need to handle any nesting. Only the « [ » branch need to handle nesting.

In fact, the proposal falls in the bad branch of « parse_Token », the « variable » branch, which tests for the presence of a « $ » char.

The good branch for such an implementation is the « command » branch, that test for the presence of a « [ » char, because a command can nest.

Conclusion : There is no real choice : the expr shorthand proposal must begin with a « [ », if not, the price paid by the source code will be too big, because of the radical difference between the « variable » and « command » concepts. At the end, you could have to recreate the flow for commands in the flow for variables.

That's why, instead, I proposed this shorthand : « [( ... )] »

It will be detected in the command branch of parse token, so that :

1. subst -noc can inhibit it : no need to change anything in the actual logic

(an open question remains : how -nocommands et -noexpression flags will interfer, we may want -nocommand -withexpression after all)

2. We just have to check

 } else if (*src == '[' && src[1] =='(') {

    if (noSubstExpr) {
        tokenPtr->type = TCL_TOKEN_TEXT;
        tokenPtr->size = 2;
        parsePtr->numTokens++;
        src+=2;
        numBytes-=2;
        continue;
        }

    Tcl_Parse *nestedPtr;   src+=2;  numBytes-=2;

    int length = tclParseExpression(src); // to be created :

   opnode *opnode

   nestedPtr = (Tcl_Parse *)TclStackAlloc(parsePtr->interp, sizeof(Tcl_Parse));

   if (Tcl_ParseExpr(interp, src, -1, nestedPtr) != TCL_OK) {...}

   ...etc

I think, globally, this would be a simpler change, since we are not mixing carrots and potatoes.

Honnestly, is the following code so bad visualy ?

% set script [subst -noc {
    set x [(1+1)]
}]
% set a 2
% set script [subst -nov {
   set x [(2+$a)]
}]
% set x [( 3 + [(3+3)] )]
% set x [expr {4+[(4+4)]}]





Le 23/10/2025 à 09:43, Jan Nijtmans a écrit :

Op do 23 okt 2025 om 07:56 schreef Florent Merlet:


What are the results of these commands:


% set script [subst -noc {
    set x $(1+1)
}]

    set x 2

% set a 2
2
% set script [subst -nov {
   set x $(2+$a)
}]

   set x 4

% set x $(3+$(3+3))
invalid character "$"
in expression "3+$(3+3)"
% set x [expr {4+$(4+4)}]
invalid character "$"
in expression "4+$(4+4)"
%

I think all of those are expected. The first answer does
"expr" substitution, the option "-noc" doesn't change
anything on that (-noexpr would!). The second one
also does expr substitution, the variable $a is handled
by the expr handling itself. So that's expected too.

The latter 2 are also as I expect, since $( substitution
should not be done within expressions itself.

This test is done with the version in the fossil "tip-672"
branch. This branch is based on an earlier version from Eric.
(@eric, could you base later changes on this version?
you use a non-standard indenting, that makes merging
in later versions rather cumbersome)

One thing already discussed was to adapt the "subst"
command for the new substitution. The tip-672 branch
adds new -expr and -noexpr options, so the new proposed
substitution can be switched on or off at will. I think
that should be added to the TIP text.

For now, I'll wait for the result of the discussion.

Florent Merlet wrote:


Is it to make people to go far away from Tcl ?


Such accusations don't help anything in this
discussion. That's all I'm going to say about it.

Hope this helps,
       Jan Nijtmans


_______________________________________________
Tcl-Core mailing list
Tcl...@li...<mailto:Tcl...@li...>
https://lists.sourceforge.net/lists/listinfo/tcl-core [lists.sourceforge.net]<https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/tcl-core__;!!PDiH4ENfjr2_Jw!G2apoGRhxYjjkcB234n_qe1j3E65OHgCeKrFVRNGPkptfDmsAOCfgWEx4K2_YpAcsWbWY_5Bm2XkftSqpPt9ZcYmA1AMGovG0wSuHcY$>


--