Re: [TCLCORE] Fwd: TIP 672 Implementation Complete - Ready for Sponsorship

[EricT <tw...@gm...>]

Actually, the only efficiency gain is with constant folding, the compiler
just can't combine the two:

% tcl::unsupported::disassemble script {set a $(1 + ([expr {2-1}]))}
ByteCode 0x23090eb7460, refCt 1, epoch 22, interp 0x230898e0830 (epoch 22)
  Source "set a $(1 + ([expr {2-1}]))"
  Cmds 3, src 27, inst 18, litObjs 3, aux 0, stkDepth 3, code/src 0.00
  Commands 3:
      1: pc 0-16, src 0-26        2: pc 5-15, src 6529-6553
      3: pc 10-14, src 6541-6550
  Command 1: "set a $(1 + ([expr {2-1}]))"
    (0) push 0 # "a"
  Command 2: "expr {1 + ([expr {2-1}])}..."
    (5) push 1 # "1"
  Command 3: "expr {2-1}..."
    (10) push 2 # "1"
    (15) add
    (16) storeStk
    (17) done

% tcl::unsupported::disassemble script {set a $(1 + (2-1))}
ByteCode 0x23090eb6c60, refCt 1, epoch 22, interp 0x230898e0830 (epoch 22)
  Source "set a $(1 + (2-1))"
  Cmds 2, src 18, inst 12, litObjs 2, aux 0, stkDepth 2, code/src 0.00
  Commands 2:
      1: pc 0-10, src 0-17        2: pc 5-9, src 3334001-3334016
  Command 1: "set a $(1 + (2-1))"
    (0) push 0 # "a"
  Command 2: "expr {1 + (2-1)}..."
    (5) push 1 # "2"
    (10) storeStk
    (11) done

E


On Thu, Oct 23, 2025 at 11:17 AM EricT <tw...@gm...> wrote:

> Jan:
>
> You mentioned the non-standard indenting - that's a tab/space mixing
> issue. I have ongoing trouble with Tcl's tab size 8 + indent 4 standard.
>
> The problem: with 8-char tabs, odd indentation levels require mixing tabs
> and spaces, which breaks when editors differ. Adding an outer if
> statement can mess up the spacing.
>
> I'd recommend Tcl adopt tab size 4 + indent 4, which eliminates the issue
> - whether your editor inserts spaces or tabs, it looks the same.
>
> For my sections: would converting to spaces-only work for the community? I
> hesitate to change the entire file.
> On testing:
>
> I converted my test file from $() to $(()) syntax and ran it against mode
> 1 - 82 of 84 passed. The two failures were empty expression tests expecting
> different error messages (become sub-expressions). I fixed them with a
> flexible pattern:
>
> string match "*empty*expression*" $msg
>
> Now they work with either syntax.  This is something to keep in mind if
> $(()) were chosen, and years later relaxed to just $() after the $(index)
> empty array was deprecated.
>
> Regarding nested $():
>
> $(... $(...)...) fails because expr (or the compiler) rejects the nested $
> in the expression.
>
> But $(... [expr {...}]...) works fine. There's no reason to nest anyway -
> removing the inner $ gives the same result more efficiently.
>
> Eric
>
>
> On Thu, Oct 23, 2025 at 8:18 AM da Silva, Peter J <
> pet...@fl...> wrote:
>
>> 5 is basically equivalent to 1, it’s just a convention.
>>
>>
>>
>> *From: *Donal Fellows <don...@ma...>
>> *Date: *Thursday, October 23, 2025 at 06:15
>> *To: *Florent Merlet <flo...@gm...>,
>> tcl...@li... <tcl...@li...>
>> *Subject: *Re: [TCLCORE] Fwd: TIP 672 Implementation Complete - Ready
>> for Sponsorship
>>
>> ​ Conclusion : There is no real choice ​ That's an incorrect conclusion.
>> At least one of the priors you use in reaching it is invalid. The core of
>> the proposal is that the sequence $( start a new syntax category (with what
>> follows being an expression
>>
>> ​
>>
>> Conclusion : There is no real choice
>>
>> ​
>>
>> That's an incorrect conclusion. At least one of the priors you use in
>> reaching it is invalid.
>>
>>
>>
>> The core of the proposal is that the sequence *$(* start a *new* syntax
>> category (with what follows being an expression to the point where there's
>> a *matching *parenthesis using an appropriate recursive matching
>> algorithm, with a similar model to how command substitutions work). This
>> new syntactic entity will be called (almost certainly) an *expression
>> substitution*. That it starts with *$* is good; existing code that
>> inserts backslashes before dollar signs to make things *subst*-safe will
>> not be surprised; normally we'd recommend such approaches instead now
>> switch to using *regsub -command*, due to the way *subst *works, but
>> that's a longer-term change-over.
>>
>>
>>
>> That the C code inside the Tcl parsing engine will need to change to
>> accommodate a new syntax category isn't a deep insight. Of course it will
>> need to change. Such change is what is being proposed!
>>
>>
>>
>> Right now, the real discussion is what the sequence of literal characters
>> in a Tcl script to introduce the new syntax should be. The options in play
>> are:
>>
>>
>>
>>    1. Do nothing. Stick with *[expr {*expression...*}]*. This option is *always
>>    open* to us.
>>
>>
>>    2. Use *$(*expression...*)*
>>
>>
>>    3. Use *$((*expression...*))*
>>
>>
>>    4. Use *[={*expression...*}]*
>>
>>
>>    5. Use *[=* expression...*]*
>>
>>
>>    6. Use *[(*expression...*)]*
>>
>>
>>    7. Use something else (I've probably missed an option or two from
>>    this painting of the bikeshed)
>>
>>
>>
>> Of the options above, 2 has *real *compatibility issues (i.e., we *know *it
>> will clash with existing code in Tcllib) and 5 probably has semantic issues
>> (because it was proposed as an actual command; anything that introduces
>> uncontrolled double substitution is a no-no at this point). Option 3 has
>> been assessed for practical compatibility, and should be fine (except for a
>> few places in JimTcl, written by someone *already in this conversation*).
>> Option 4 manages to be ugly, but is probably OK in terms of amount of
>> in-the-wild usage. Option 6 hasn't been assessed yet.
>>
>>
>>
>> For all options that introduce a new syntactic substitution class, a
>> change to *subst *is needed to work with it (Jan's outlined this
>> adequately) and there should be consideration whether the other
>> mini-languages inside Tcl should change (we have several).
>>
>>
>>
>> Donal.
>>
>>
>> ------------------------------
>>
>> *From:* Florent Merlet <flo...@gm...>
>> *Sent:* Thursday, October 23, 2025 09:32
>> *To:* tcl...@li... <tcl...@li...>
>> *Subject:* Re: [TCLCORE] Fwd: TIP 672 Implementation Complete - Ready
>> for Sponsorship
>>
>>
>>
>> Hello Jan, So we can admit that the implementation is not ready. In fact,
>> expression is modelised like a variable and it can't nest. Fondamentally,
>> the problem is that the « $ » sign allways has been related to the concept
>> of variable, whereas
>>
>> Hello Jan,
>>
>> So we can admit that the implementation is not ready.
>>
>> In fact, expression is modelised like a variable and it can't nest.
>>
>> Fondamentally, the problem is that the « $ » sign allways has been
>> related to the concept of variable, whereas an expression is related to the
>> concept of command.
>>
>> « variables » have caracteristics that differ from the one of « commands
>> » .
>>
>> « commands » can nest, whereas « variables » don't.
>>
>> These distinct caracteristics were translated directly in the programm
>> flow, because, at a certain degree, there must be an analogy between the
>> flow of the programm and the caracteristics of the concept.
>>
>> In Parse_token, the « $ » branch doesn't need to handle any nesting. Only
>> the « [ » branch need to handle nesting.
>>
>> In fact, the proposal falls in the bad branch of « parse_Token », the «
>> variable » branch, which tests for the presence of a « $ » char.
>>
>> The good branch for such an implementation is the « command »
>> branch, that test for the presence of a « [ » char, because a command can
>> nest.
>>
>> Conclusion : There is no real choice : the expr shorthand proposal must
>> begin with a « [ », if not, the price paid by the source code will be too
>> big, because of the radical difference between the « variable » and «
>> command » concepts. At the end, you could have to recreate the flow for
>> commands in the flow for variables.
>>
>> That's why, instead, I proposed this shorthand : « [( ... )] »
>>
>> It will be detected in the command branch of parse token, so that :
>>
>> 1. subst -noc can inhibit it : no need to change anything in the actual
>> logic
>>
>> (an open question remains : how -nocommands et -noexpression flags will
>> interfer, we may want -nocommand -withexpression after all)
>>
>> 2. We just have to check
>>
>>  } else if (*src == '[' && src[1] =='(') {
>>
>>     if (noSubstExpr) {
>>         tokenPtr->type = TCL_TOKEN_TEXT;
>>         tokenPtr->size = 2;
>>         parsePtr->numTokens++;
>>         src+=2;
>>         numBytes-=2;
>>         continue;
>>         }
>>
>>     Tcl_Parse *nestedPtr;   src+=2;  numBytes-=2;
>>
>>     int length = tclParseExpression(src); // to be created :
>>
>>    opnode *opnode
>>
>>    nestedPtr = (Tcl_Parse *)TclStackAlloc(parsePtr->interp,
>> sizeof(Tcl_Parse));
>>
>>    if (Tcl_ParseExpr(interp, src, -1, nestedPtr) != TCL_OK) {...}
>>
>>    ...etc
>>
>> I think, globally, this would be a simpler change, since we are not mixing carrots and potatoes.
>>
>> Honnestly, is the following code so bad visualy ?
>>
>> % set script [subst -noc {
>>
>>     set x [(1+1)]
>>
>> }]
>>
>> % set a 2
>>
>> % set script [subst -nov {
>>
>>    set x [(2+$a)]
>>
>> }]
>>
>> % set x [( 3 + [(3+3)] )]
>>
>> % set x [expr {4+[(4+4)]}]
>>
>>
>>
>>
>>
>>
>>
>> Le 23/10/2025 à 09:43, Jan Nijtmans a écrit :
>>
>> Op do 23 okt 2025 om 07:56 schreef Florent Merlet:
>>
>> What are the results of these commands:
>>
>> % set script [subst -noc {
>>
>>     set x $(1+1)
>>
>> }]
>>
>>
>>
>>     set x 2
>>
>>
>>
>> % set a 2
>>
>> 2
>>
>> % set script [subst -nov {
>>
>>    set x $(2+$a)
>>
>> }]
>>
>>
>>
>>    set x 4
>>
>>
>>
>> % set x $(3+$(3+3))
>>
>> invalid character "$"
>>
>> in expression "3+$(3+3)"
>>
>> % set x [expr {4+$(4+4)}]
>>
>> invalid character "$"
>>
>> in expression "4+$(4+4)"
>>
>> %
>>
>>
>>
>> I think all of those are expected. The first answer does
>>
>> "expr" substitution, the option "-noc" doesn't change
>>
>> anything on that (-noexpr would!). The second one
>>
>> also does expr substitution, the variable $a is handled
>>
>> by the expr handling itself. So that's expected too.
>>
>>
>>
>> The latter 2 are also as I expect, since $( substitution
>>
>> should not be done within expressions itself.
>>
>>
>>
>> This test is done with the version in the fossil "tip-672"
>>
>> branch. This branch is based on an earlier version from Eric.
>>
>> (@eric, could you base later changes on this version?
>>
>> you use a non-standard indenting, that makes merging
>>
>> in later versions rather cumbersome)
>>
>>
>>
>> One thing already discussed was to adapt the "subst"
>>
>> command for the new substitution. The tip-672 branch
>>
>> adds new -expr and -noexpr options, so the new proposed
>>
>> substitution can be switched on or off at will. I think
>>
>> that should be added to the TIP text.
>>
>>
>>
>> For now, I'll wait for the result of the discussion.
>>
>>
>>
>> Florent Merlet wrote:
>>
>> Is it to make people to go far away from Tcl ?
>>
>> Such accusations don't help anything in this
>>
>> discussion. That's all I'm going to say about it.
>>
>> Hope this helps,
>>
>>        Jan Nijtmans
>>
>>
>>
>>
>>
>> _______________________________________________
>>
>> Tcl-Core mailing list
>>
>> Tcl...@li...
>>
>> https://lists.sourceforge.net/lists/listinfo/tcl-core [lists.sourceforge.net] <https://urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/tcl-core__;!!PDiH4ENfjr2_Jw!G2apoGRhxYjjkcB234n_qe1j3E65OHgCeKrFVRNGPkptfDmsAOCfgWEx4K2_YpAcsWbWY_5Bm2XkftSqpPt9ZcYmA1AMGovG0wSuHcY$>
>>
>> --
>>
>>
>> _______________________________________________
>> Tcl-Core mailing list
>> Tcl...@li...
>> https://lists.sourceforge.net/lists/listinfo/tcl-core
>>
>