tcl-core — Tcl Core Team discussions about Tcl development

Re: [TCLCORE] On namespace unknown and TIP 689

[Poor Y. <org...@po...>]

On 2025-08-02 08:53, apnmbx-public--- via Tcl-Core wrote:
> Namespace ensembles have some benefits in flexibility but the argument
> [namespace ensemble] is "beautiful" is of course only an opinion, not a
> technical argument. If the ability to rearrange arguments etc. is not
> needed, it is perfectly reasonable for someone to prefer encapsulating
> related commands using namespaces or even TclOO-based object commands.
> Speaking for myself, I generally prefer a "command data data data" 
> syntax
> where the data arguments are clearly distinguished from the actual 
> command.
> But that is just my preference, not because I think "command subcommand 
> data
> data" is somehow wrong.
> 
> And with regards to performance, the two are comparable only for
> core-compiled commands. I'd logged a ticket on this a long while back 
> where
> someone (I think dkf) explained why it was either infeasible or a
> significant amount of work to make non-core ensembles as efficient.
> 
> Compare the following. In effect, namespace invocations are 2.5x faster 
> than
> ensemble invocations. You may dismiss this as "fractional microseconds" 
> but
> microseconds are an eternity in modern processors. To put it 
> differently, as
> the number below show it is comparable to adding a proc invocation 
> wrapper
> around the actual command on every invocation.
> 
> % timerate {tarray::column size $C}
> 0.374062 µs/# 2434512 # 2673351 #/sec 910.659 net-ms
> % timerate {tarray::column::size $C}
> 0.159862 µs/# 5087524 # 6255401 #/sec 813.301 net-ms
> % proc p {} {}
> % timerate p
> 0.185536 µs/# 4503379 # 5389782 #/sec 835.540 net-ms
> 
> In any case, this is just a theoretic argument at this point. I 
> suggested
> "expando" as an alternative to achieve 689 goals if someone feels it
> appropriate and the use case sufficiently important. I myself have 
> other
> priorities.
> 
> /Ashok


I didn't argue that [namespace ensemble] is beautiful.  I argued that
the [namespace ensemble] implementation of the clock initialization
routines was beautiful, and I think that it clearly is, compared to
the tcl::clock::uninitialized contortions that it replaces.  We all
know that using features in unintended hacky ways can be ugly.

I'm not dismissing the performance argument, only saying that
addressing it directly is better than hacking in a workaround and then
making it a permanent feature of Tcl.  I think the performance issue
can be resolved.  Furthermore, I think that the more dynamic features
of namespaces were intentionally isolated into the [namespace
ensemble] layer in order to avoid performance issues with the
foundational namespace features.  There is a runtime cost to things
like expando.

Don't avoid [namespace ensemble] when it was provided for
solving the very problem you have.

--
Yorick

Re: [TCLCORE] On namespace unknown and TIP 689

[<apn...@ya...>]

Namespace ensembles have some benefits in flexibility but the argument
[namespace ensemble] is "beautiful" is of course only an opinion, not a
technical argument. If the ability to rearrange arguments etc. is not
needed, it is perfectly reasonable for someone to prefer encapsulating
related commands using namespaces or even TclOO-based object commands.
Speaking for myself, I generally prefer a "command data data data" syntax
where the data arguments are clearly distinguished from the actual command.
But that is just my preference, not because I think "command subcommand data
data" is somehow wrong.

And with regards to performance, the two are comparable only for
core-compiled commands. I'd logged a ticket on this a long while back where
someone (I think dkf) explained why it was either infeasible or a
significant amount of work to make non-core ensembles as efficient.

Compare the following. In effect, namespace invocations are 2.5x faster than
ensemble invocations. You may dismiss this as "fractional microseconds" but
microseconds are an eternity in modern processors. To put it differently, as
the number below show it is comparable to adding a proc invocation wrapper
around the actual command on every invocation.

% timerate {tarray::column size $C}
0.374062 µs/# 2434512 # 2673351 #/sec 910.659 net-ms
% timerate {tarray::column::size $C}
0.159862 µs/# 5087524 # 6255401 #/sec 813.301 net-ms
% proc p {} {}
% timerate p
0.185536 µs/# 4503379 # 5389782 #/sec 835.540 net-ms

In any case, this is just a theoretic argument at this point. I suggested
"expando" as an alternative to achieve 689 goals if someone feels it
appropriate and the use case sufficiently important. I myself have other
priorities.

/Ashok

-----Original Message-----
From: Poor Yorick <org...@po...> 
Sent: Friday, August 1, 2025 7:03 PM
To: tcl...@li...
Subject: Re: [TCLCORE] On namespace unknown and TIP 689

On 2025-08-01 15:46, Dipl. Ing. Sergey G. Brester wrote:
> 01.08.2025 07:43, Poor Yorick wrote:
> 
>> Tcl already has such a feature, and it's even already got a reasonable
>> 
>> name: [namespace ensemble]. This stuff was all already thought-out,
>> discussed, and added a long time ago. [namespace ensemble] exists to
>> provide the more dynamic features of a namespace. [clock] is a
>> namespace ensemble command, and [namespace ensemble] is a beautiful
>> way to implement it. Sergei doesn't want to use [namespace ensemble]
>> because apparently it takes a microsecond or two longer than the way
>> he came up with. In TIP 689 I there is already a link to the
>> implementation of clock that I provided, showing that this can already
>> be done more cleanly with [namespace ensemble]. A new "expando"
>> procedure would amount to a second implementation of [namespace
>> ensemble] phrased slightly differently, and the only benefit would be
>> that the new idiom would currently be microscopically faster than the
>> [namespace ensemble] way. I suggest focusing on optimizing [namespace
>> ensemble] instead. Tcl should ditch the namespace-ensemble-avoiding
>> implementation of the clock initialization routine, and should not
>> reimplement [namespace ensemble] as [expando]. Doing so would
>> enshrine a hack that was used for dubious reasons as part of the Tcl
>> public interface, and lead to endless questions about whether to use
>> [namespace ensemble] or [expando].



> You're trying to mix bananas with cucumbers, again...
> You might wonder, but THERE ARE NAMESPACE WITHOUT ENSEMBLES! A LOT!
> I'd even say dozen times more than ensembles, percentage-wise seen.
> 
> It is as if a clear example was provided in C, so without classes,
> and then someone (an internet troll?) asking why don't use use C++ and
> classes.
> 
> So, please, be objective and stop flooding with arguments that aren't
> arguments.


Here is the argument: The purpose of [namespace ensemble] is to
provide a dynamically-configurable public interface for namespaces.
[clock] needs exactly that for the late loading of some of its
ensemble commands. [namespace ensemble] provides a facility to
accomplish exactly the functionality that you want to accomplish:
Bring a procedure into existence when the caller requests it. Here is
an implementation of late initialization of clock ensemble commands
using [namespace ensemble]:

      https://core.tcl-lang.org/tcl/info/ed50d4c0b3c7782e

This [namespace ensemble] implementation for [clock] is
straightforward, and uses namespaces and namespace ensembles as
intended.  The assumption of TIP 689 is that [namespace ensemble]
isn't good enough, and that therefore another facility should exist.
But why?  If the need for more dynamic namespace configuration was
foreseen, and [namespace ensemble] provided for that purpose, why add
yet another mechanism for doing the same thing?  So far, the only
rationale I've seen in the discussions for that is that "namespace
ensemble is too slow".  If that's true it could be addressed directly
instead of trying to work around it.

--
Yorick


_______________________________________________
Tcl-Core mailing list
Tcl...@li...
https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] On namespace unknown and TIP 689

[Poor Y. <org...@po...>]

On 2025-08-01 15:46, Dipl. Ing. Sergey G. Brester wrote:
> 01.08.2025 07:43, Poor Yorick wrote:
> 
>> Tcl already has such a feature, and it's even already got a reasonable
>> 
>> name: [namespace ensemble]. This stuff was all already thought-out,
>> discussed, and added a long time ago. [namespace ensemble] exists to
>> provide the more dynamic features of a namespace. [clock] is a
>> namespace ensemble command, and [namespace ensemble] is a beautiful
>> way to implement it. Sergei doesn't want to use [namespace ensemble]
>> because apparently it takes a microsecond or two longer than the way
>> he came up with. In TIP 689 I there is already a link to the
>> implementation of clock that I provided, showing that this can already
>> be done more cleanly with [namespace ensemble]. A new "expando"
>> procedure would amount to a second implementation of [namespace
>> ensemble] phrased slightly differently, and the only benefit would be
>> that the new idiom would currently be microscopically faster than the
>> [namespace ensemble] way. I suggest focusing on optimizing [namespace
>> ensemble] instead. Tcl should ditch the namespace-ensemble-avoiding
>> implementation of the clock initialization routine, and should not
>> reimplement [namespace ensemble] as [expando]. Doing so would
>> enshrine a hack that was used for dubious reasons as part of the Tcl
>> public interface, and lead to endless questions about whether to use
>> [namespace ensemble] or [expando].



> You're trying to mix bananas with cucumbers, again...
> You might wonder, but THERE ARE NAMESPACE WITHOUT ENSEMBLES! A LOT!
> I'd even say dozen times more than ensembles, percentage-wise seen.
> 
> It is as if a clear example was provided in C, so without classes,
> and then someone (an internet troll?) asking why don't use use C++ and
> classes.
> 
> So, please, be objective and stop flooding with arguments that aren't
> arguments.


Here is the argument: The purpose of [namespace ensemble] is to
provide a dynamically-configurable public interface for namespaces.
[clock] needs exactly that for the late loading of some of its
ensemble commands. [namespace ensemble] provides a facility to
accomplish exactly the functionality that you want to accomplish:
Bring a procedure into existence when the caller requests it. Here is
an implementation of late initialization of clock ensemble commands
using [namespace ensemble]:

      https://core.tcl-lang.org/tcl/info/ed50d4c0b3c7782e

This [namespace ensemble] implementation for [clock] is
straightforward, and uses namespaces and namespace ensembles as
intended.  The assumption of TIP 689 is that [namespace ensemble]
isn't good enough, and that therefore another facility should exist.
But why?  If the need for more dynamic namespace configuration was
foreseen, and [namespace ensemble] provided for that purpose, why add
yet another mechanism for doing the same thing?  So far, the only
rationale I've seen in the discussions for that is that "namespace
ensemble is too slow".  If that's true it could be addressed directly
instead of trying to work around it.

--
Yorick

Re: [TCLCORE] On namespace unknown and TIP 689

[Dipl. I. S. G. B. <se...@us...>]

 

You're trying to mix bananas with cucumbers, again...
You might wonder, but THERE ARE NAMESPACE WITHOUT ENSEMBLES! A LOT! 
I'd even say dozen times more than ensembles, percentage-wise seen. 

It is as if a clear example was provided in C, so without classes, 
and then someone (an internet troll?) asking why don't use use C++ and
classes. 

So, please, be objective and stop flooding with arguments that aren't
arguments. 

Regards,
Serg. 

01.08.2025 07:43, Poor Yorick wrote: 

> Tcl already has such a feature, and it's even already got a reasonable 
> 
> name: [namespace ensemble]. This stuff was all already thought-out,
> discussed, and added a long time ago. [namespace ensemble] exists to
> provide the more dynamic features of a namespace. [clock] is a
> namespace ensemble command, and [namespace ensemble] is a beautiful
> way to implement it. Sergei doesn't want to use [namespace ensemble]
> because apparently it takes a microsecond or two longer than the way
> he came up with. In TIP 689 I there is already a link to the
> implementation of clock that I provided, showing that this can already
> be done more cleanly with [namespace ensemble]. A new "expando"
> procedure would amount to a second implementation of [namespace
> ensemble] phrased slightly differently, and the only benefit would be
> that the new idiom would currently be microscopically faster than the
> [namespace ensemble] way. I suggest focusing on optimizing [namespace
> ensemble] instead. Tcl should ditch the namespace-ensemble-avoiding
> implementation of the clock initialization routine, and should not
> reimplement [namespace ensemble] as [expando]. Doing so would
> enshrine a hack that was used for dubious reasons as part of the Tcl
> public interface, and lead to endless questions about whether to use
> [namespace ensemble] or [expando].
> 
> --
> 
> Yorick
> 
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core [1]
 

Links:
------
[1] https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] On namespace unknown and TIP 689

[Poor Y. <org...@po...>]

On 2025-07-31 16:38, da Silva, Peter J wrote:
> Looks good except that “expando” is obviously a Fantastic Four villain. 
> ☺
> 
> From: apnmbx-public--- via Tcl-Core <tcl...@li...>
n> Date: Tuesday, July 29, 2025 at 01:11
> To: tcl...@li... <tcl...@li...>
> Subject: [TCLCORE] On namespace unknown and TIP 689
> There has been an ongoing debate every few months about the behavior of 
> namespace unknown and TIP 689. Sergey recently also updated a ticket on 
> the matter (if I understood correctly) on benefits in performance 
> through delayed package loading. 
> 
> There has been an ongoing debate every few months about the behavior of 
> namespace unknown and TIP 689. Sergey recently also updated a ticket on 
> the matter (if I understood correctly) on benefits in performance 
> through delayed package loading.
> 
> I understand the pros of TIP 689 but am also cognizant of 
> incompatibilities in unforeseen ways, whether real or perceived.
> 
> I wonder if the following proposal would satisfy both parties by 
> allowing dynamic addition of commands to a namespace while greatly 
> reducing (but probably not completely eliminating) the chance of 
> incompatibility.
> 
> A new command is introduced. (“expando” is a placeholder coming from 
> COM terminology)
> 
> namespace eval ns {
>     namespace expando expandoProc
>     proc expandoProc commandName {
>         load or otherwise make commandName available and return 1
>         otherwise return 0
>     }
> }
> 
> namespace ns2 {
>    ::ns::someProc
> }
> 
> The Tcl command resolution is modified as follows.
> 
>   *   The command ns::someProc is looked up in the current fashion 
> (namespace path etc.).
>   *   If not found, the *target* namespace ns is checked if it has 
> defined an expandoProc. Since it has, it is called. If it returns 1 
> indicating it created a someProc command in its namespace, the original 
> command is re-executed.
>   *   If the ns had not defined an expandoProc or if expandoProc 
> returned 0, the current namespace unknown algorithm is followed.
> 
> With the clock command for example, no preloading needs to be done. The 
> first time clock::format is called, clock’s expando proc will load the 
> implementation.
> 
> At the same time, since namespaces explicitly opt-in as expandable, the 
> incompatibilities will be intended and limited.
> 
> Obviously much hand-waving above and details to be sorted out.
> 
> But any obvious (or otherwise) flaws in the above? Thoughts?

Tcl already has such a feature, and it's even already got a reasonable
name: [namespace ensemble].  This stuff was all already thought-out,
discussed, and added a long time ago. [namespace ensemble] exists to
provide the more dynamic features of a namespace. [clock] is a
namespace ensemble command, and [namespace ensemble] is a beautiful
way to implement it.  Sergei doesn't want to use [namespace ensemble]
because apparently it takes a microsecond or two longer than the way
he came up with.  In TIP 689 I there is already a link to the
implementation of clock that I provided, showing that this can already
be done more cleanly with [namespace ensemble].  A new "expando"
procedure would amount to a second implementation of [namespace
ensemble] phrased slightly differently, and the only benefit would be
that the new idiom would currently be microscopically faster than the
[namespace ensemble] way.  I suggest focusing on optimizing [namespace
ensemble] instead.  Tcl should ditch the namespace-ensemble-avoiding
implementation of the clock initialization routine, and should not
reimplement [namespace ensemble] as [expando].  Doing so would
enshrine a hack that was used for dubious reasons as part of the Tcl
public interface, and lead to endless questions about whether to use
[namespace ensemble] or [expando].

--

Yorick

Re: [TCLCORE] On namespace unknown and TIP 689

[da S. P. J <pet...@fl...>]

Looks good except that “expando” is obviously a Fantastic Four villain. ☺

From: apnmbx-public--- via Tcl-Core <tcl...@li...>
Date: Tuesday, July 29, 2025 at 01:11
To: tcl...@li... <tcl...@li...>
Subject: [TCLCORE] On namespace unknown and TIP 689
There has been an ongoing debate every few months about the behavior of namespace unknown and TIP 689. Sergey recently also updated a ticket on the matter (if I understood correctly) on benefits in performance through delayed package loading. 

There has been an ongoing debate every few months about the behavior of namespace unknown and TIP 689. Sergey recently also updated a ticket on the matter (if I understood correctly) on benefits in performance through delayed package loading.

I understand the pros of TIP 689 but am also cognizant of incompatibilities in unforeseen ways, whether real or perceived.

I wonder if the following proposal would satisfy both parties by allowing dynamic addition of commands to a namespace while greatly reducing (but probably not completely eliminating) the chance of incompatibility.

A new command is introduced. (“expando” is a placeholder coming from COM terminology)

namespace eval ns {
    namespace expando expandoProc
    proc expandoProc commandName {
        load or otherwise make commandName available and return 1
        otherwise return 0
    }
}

namespace ns2 {
   ::ns::someProc
}

The Tcl command resolution is modified as follows.

  *   The command ns::someProc is looked up in the current fashion (namespace path etc.).
  *   If not found, the *target* namespace ns is checked if it has defined an expandoProc. Since it has, it is called. If it returns 1 indicating it created a someProc command in its namespace, the original command is re-executed.
  *   If the ns had not defined an expandoProc or if expandoProc returned 0, the current namespace unknown algorithm is followed.

With the clock command for example, no preloading needs to be done. The first time clock::format is called, clock’s expando proc will load the implementation.

At the same time, since namespaces explicitly opt-in as expandable, the incompatibilities will be intended and limited.

Obviously much hand-waving above and details to be sorted out.

But any obvious (or otherwise) flaws in the above? Thoughts?

/Ashok

[TCLCORE] File system epoch

[<apn...@ya...>]

Sergey,

 

Looking into your question below about why the file system epoch is incremented more often in 9.x (test suite run -> 1300 for 9.0, less than 200 for 8.6), it appears to happen when an interp is created (whether in the same thread or a different one). Creation of a new interp -> search for init.tcl in zipfs -> ScriptLibrarySetup which sets up new encodings path and calls TclpSetInitialEncodings (see bug [fccb9f322f]). This last also results in file system epoch being bumped up. As a separate bug I just logged ([87b69745be]), it also resets the encodings search path.

 

The right fix I think would be to move zipfs discovery and initialization earlier in the interpreter init sequence but interp initialization is quite a convoluted mystery to me, jumping back and forth between script and C with recursive calls to the same functions(s). I am hesitant to mess around with that. As a less risky point fix, I am thinking of changing ScriptLibrarySetup to (a) not overwrite existing state of [encoding dirs] but only add it to it (fix for 87b6) and (b) not bump the file system epoch unless the encoding search paths have changed.

 

What do you (or anyone else who knows this init sequence) think?

 

/Ashok

 

 

From: Dipl. Ing. Sergey G. Brester via Tcl-Core <tcl...@li...> 

But the question (3) from my initial mail remains, why it is so flashy in 9.x (but not in 8.x), if that use-after-free is so old and exists there since dozen years?

 

[TCLCORE] Tk 9.1a0 RELEASED

[Donald G P. <don...@ni...>]

Tk 9.1a0 Release Announcement
==============================

July 30, 2025

The Tcl Core Team is pleased to announce the release of Tk 9.1a0.
This is the first alpha release of Tk 9.1.

The Tk Toolkit is an extension to Tcl, providing commands and supports
for the creation of graphical user interfaces.  Tk originates with John
Ousterhout and his team at U.C. Berkeley in the late 1980s.  Its development
is continued by the efforts of a global network of volunteers guided by
the Tcl Core Team.

We would like to express our gratitude to all those who submit bug
reports and patches.  This information is invaluable in enabling us
to identify and eliminate problems. Such reports can be submitted here.

 > [Tk Ticket Tracker](https://core.tcl-lang.org/tk/ticket)

We ask that you log in (anonymous if you wish) to create tickets.
This deters abuse of the ticketing system:

 > [Tk Contributor Login](https://core.tcl-lang.org/tk/login)

Where to get the new releases
=============================

Tk 9.1a0 sources are freely available as open source from the Tcl
SourceForge project's file distribution area:

 > [Tk Source Distribution](https://sourceforge.net/projects/tcl/files/)

The Tk 9.1a0 distribution is source code only.  We keep links to some
third parties offering pre-built binaries for various systems here:

 > [Tk Binary Distribution](https://www.tcl-lang.org/software/tcltk/bindist.html)

Tk 9.1 Release Summary
=======================

This is a new minor version of Tk 9.  When compared with the prior release
Tk 9.0, there are new features and interfaces.  Tk 9.1 should remain
compatible with scripts and packages written to the public interfaces of
Tk 9.0.  A summary of the most noteworthy changes is found below.

Tcl Improvement Proposals (TIPs)
================================

Each new user-visible feature in Tk should find its origins in a Tcl
Improvement Proposal (TIP).  TIPs are published, edited, considered and
voted in public, and should contain valuable information about how a
feature came to be the way it is.  See the full collection here:

 > [TIP Index](https://tip.tcl-lang.org/)

Tk Changes Summary
===================

(from changes.md in the source code distribution)

The source code for Tk is managed by fossil.  Tk developers coordinate all
changes to the Tk source code at

 > [Tk Source Code](https://core.tcl-lang.org/tk/)

Release Tk 9.1a0 arises from the check-in with tag `core-9-1-a0`.

Tk 9.1a0 continues the Tk 9.x series of releases.  The Tk 9.x series
do not support Tcl 8.6.  The Tk 9.1 series extends the Tcl 9.0 series.
To make use of Tk 9.1a0, first a Tcl 9.0 or 9.1 release must be present.
As new Tk features are developed, expect them to appear in Tk 9, but not
necessarily in Tk 8.

# 9.1 Features and Interfaces
  - [MS-Win: remove Windows XP dialog variants for tk_chooseDirectory and tk_getOpenFile](https://core.tcl-lang.org/tk/tktview/441c52)
  - [Handle negative screen distances](https://core.tcl-lang.org/tips/doc/trunk/tip/698.md)
  - [Extend Tk_CanvasTextInfo](https://core.tcl-lang.org/tips/doc/trunk/tip/704.md)
  - [Add new states to ttk::treeview and ttk::notebook](https://core.tcl-lang.org/tips/doc/trunk/tip/719.md)

# Potential incompatibilities to 9.0
  - [MS-Win: the undocumented option -xpstyle was removed from tk_chooseDirectory and tk_getOpenFile](https://core.tcl-lang.org/tk/tktview/441c52)

For additional information:
===========================

Please visit the Tcl Developer Xchange web site:

 > [Tcl Developer Xchange](https://www.tcl-lang.org/)

This site contains a variety of information about Tcl/Tk in general, the
core Tcl and Tk distributions, Tcl development tools, and much more.

--
Tcl Core Team and Maintainers
Don Porter, Tcl Core Release Manager

-- 
| Don Porter            Applied and Computational Mathematics Division |
| don...@ni...             Information Technology Laboratory |
| http://math.nist.gov/~DPorter/                                  NIST |
|______________________________________________________________________|

[TCLCORE] Tcl 9.1a0 RELEASED

[Donald G P. <don...@ni...>]

Tcl 9.1a0 Release Announcement
==============================

July 30, 2025

The Tcl Core Team is pleased to announce the release of Tcl 9.1a0.
This is the first alpha release of Tcl 9.1.

Tcl is the Tool Command Language originated by John Ousterhout and his
team at U.C. Berkeley in the late 1980s.  Its development is continued
by the efforts of a global network of volunteers guided by the Tcl Core Team.

We would like to express our gratitude to all those who submit bug
reports and patches.  This information is invaluable in enabling us
to identify and eliminate problems. Such reports can be submitted here.

 > [Tcl Ticket Tracker](https://core.tcl-lang.org/tcl/ticket)

We ask that you log in (anonymous if you wish) to create tickets.
This deters abuse of the ticketing system:

 > [Tcl Contributor Login](https://core.tcl-lang.org/tcl/login)

Where to get the new releases
=============================

Tcl 9.1a0 sources are freely available as open source from the Tcl
SourceForge project's file distribution area:

 > [Tcl Source Distribution](https://sourceforge.net/projects/tcl/files/)

The Tcl 9.1a0 distribution is source code only.  We keep links to some
third parties offering pre-built binaries for various systems here:

 > [Tcl Binary Distribution](https://www.tcl-lang.org/software/tcltk/bindist.html)

Tcl Summary
===========

The Tcl distribution delivers C source code that builds into a C library
providing interpreters and related supports to execute programs written
in the Tcl programming language.  Source code for the application program
`tclsh` is also included.  `tclsh` provides a shell for either interactive
execution of Tcl commands, or execution of files containing Tcl programs.

Tcl is an extensible language, and the Tcl C library provides interfaces
for the creation of extension libraries adding new commands and features
to the core Tcl command set.  Tcl 9 debuts the full feature set needed to
package an application written in C and Tcl into a single file executable
exploiting virtual filesystem archives.

Tcl 9.1 Release Summary
=======================

This is a new minor version of Tcl 9.  When compared with the prior release
Tcl 9.0, there are new features and interfaces.  Tcl 9.1 should remain
compatible with scripts and packages written to the public interfaces of
Tcl 9.0.  A summary of the most noteworthy changes is found below.

Tcl Improvement Proposals (TIPs)
================================

Each new user-visible feature in Tcl should find its origins in a Tcl
Improvement Proposal (TIP).  TIPs are published, edited, considered and
voted in public, and should contain valuable information about how a
feature came to be the way it is.  See the full collection here:

 > [TIP Index](https://tip.tcl-lang.org/)

Tcl Changes Summary
===================

(from changes.md in the source code distribution)

The source code for Tcl is managed by fossil.  Tcl developers coordinate all
changes to the Tcl source code at

 > [Tcl Source Code](https://core.tcl-lang.org/tcl/timeline)

Release Tcl 9.1a0 arises from the check-in with tag `core-9-1-a0`.

Highlighted differences between Tcl 9.1 and Tcl 9.0 are summarized below,
with focus on changes important to programmers using the Tcl library and
writing Tcl scripts.

# New commands and options

- [New options -backslashes, -commands and -variables for subst command](https://core.tcl-lang.org/tips/doc/trunk/tip/712.md)

# New public C API

- [Tcl\_IsEmpty checks if the string representation of a value would be the empty string](https://core.tcl-lang.org/tips/doc/trunk/tip/711.md)
- [Tcl\_GetEncodingNameForUser returns name of encoding from user settings](https://core.tcl-lang.org/tips/doc/trunk/tip/716.md)
- [Tcl\_AttemptCreateHashEntry - version of Tcl\_CreateHashEntry that returns NULL instead of panic'ing on memory allocation errors](https://core.tcl-lang.org/tips/doc/trunk/tip/717.md)
- [Tcl\_ListObjRange, Tcl\_ListObjRepeat, Tcl\_TclListObjReverse - C API for new list operations](https://core.tcl-lang.org/tips/doc/trunk/tip/649.md)

# Performance

- [Memory efficient internal representations](https://core.tcl-lang.org/tcl/wiki?name=New+abstract+list+representations)
for list operations on large lists.

# Bug fixes
  - [tclEpollNotfy PlatformEventsControl panics if websocket disconnected](https://core.tcl-lang.org/tcl/tktview/010d8f38)

For additional information:
===========================

Please visit the Tcl Developer Xchange web site:

 > [Tcl Developer Xchange](https://www.tcl-lang.org/)

This site contains a variety of information about Tcl/Tk in general, the
core Tcl and Tk distributions, Tcl development tools, and much more.

--
Tcl Core Team and Maintainers
Don Porter, Tcl Core Release Manager

-- 
| Don Porter            Applied and Computational Mathematics Division |
| don...@ni...             Information Technology Laboratory |
| http://math.nist.gov/~DPorter/                                  NIST |
|______________________________________________________________________|

[TCLCORE] Tcl 9.1a0 Release Candidate

[Donald G P. <don...@ni...>]

Now available from

https://sourceforge.net/projects/tcl/files/Tcl/9.1a0/

is the RC3 release candidate of the first alpha release of Tcl 9.1.

This is the fourth of a sequence of candidate releases leading to the release of
Tcl 9.1a0.

Compared to RC2, the only changes are edits to the release notes, to one code comment,
and the revision of some tests in the test suite.  I think there is no need to repeat testing
done on RC2.  There is no functional change.

Not all release artifacts have been updated to RC3 names, as their content did not change.

I am moving directly toward making these files the release.

Thank you for your contributions and assistance.

Re: [TCLCORE] Tcl 9.1a0 Release Candidate

[Pietro C. <ga...@ga...>]

On Jul 29 2025, 19:39 +0000, Donald G Porter via Tcl-Core <tcl...@li...> wrote:
>
>
>Now available from
>
>https://sourceforge.net/projects/tcl/files/Tcl/9.1a0/
>
>is the RC2 release candidate of the first alpha release of Tcl 9.1.

All good on FreeBSD's side.

Thanks!

-- 
Pietro Cerutti
I have pledged to give 10% of income to effective charities
and invite you to join me - https://givingwhatwecan.org

Re: [TCLCORE] Tcl 9.1a0 Release Candidate

[<apn...@ya...>]

TIP 649 is missing (the internal list rep bullet is separate) as is TIP 712. 
Have committed changes.md to trunk with bullets for them.

Verified default builds for Tcl/Tk + pkgs (except database drivers) on Windows 10 Visual Studio 2022 (x64 and x86) and Tcl+pkgs (without Tk) on Ubuntu 20 (WSL)

Re: [TCLCORE] Tcl 9.1a0 Release Candidate

[Harald O. <har...@el...>]

Am 29.07.2025 um 21:39 schrieb Donald G Porter via Tcl-Core:
> Release notes are also available for review and correction.
Great release notes. Some ideas:

Add "by avoiding shimmering in many cases" behind "Tcl\_IsEmpty checks 
if the string representation of a value would be the empty string"

Add TIP 712

I would add the byte code improvements at the performance section.

I would not mention the bug fix, as this is also fixed in 9.0.2.

---

Tk:

I would not mention the Tk 9.0 points:
- MS-Win removal of Windows XP, and removal of "-xpstyle".

TIP 725 is missing.

Thanks for all,
Harald

Re: [TCLCORE] Tcl 9.1a0 Release Candidate

[Harald O. <har...@el...>]

Am 29.07.2025 um 21:39 schrieb Donald G Porter via Tcl-Core:
> 
> 
> Now available from
> 
> https://sourceforge.net/projects/tcl/files/Tcl/9.1a0/
> 
> is the RC2 release candidate of the first alpha release of Tcl 9.1.

Tested on Win 64 GER with MS-VS2022 64 bit and makefile.vc build:
- compile, test, htmlhelp, install, test with huge application: all ok.

Thanks for the great work,
Harald

Re: [TCLCORE] FW: On namespace unknown and TIP 689

[Dipl. I. S. G. B. <se...@us...>]

 

Sure, the questions weren't directed at you...

However I asked about particular use case or an example where the new
behavior could bother.
Neither your example illustrate it, nor it looks to me as a real use
case, for several reasons: 

 	* it'd would surely work further, unless it is something really
artificial to produce the "incompatibility" intentionally.

	* what shall happen if the ::platform::generic shall be loaded outside
of the NS (in different way as I understand)? 
or if the command already exists, as known (loaded outside before)? 
and especially why a lazy loading for ::platform::generic shall be
registered for namespace pkg1 (together with another handler)? 
	* if two handlers would be specified, why they must do different things
(as you said to conflict) to load ::platform::generic? 
and isn't it the contradiction already there? just imagine another would
be invoked before yours (completely without TIP#689): 

namespace eval another-pkg {::platform::generic}
namespace eval pkg1 {::platform::generic}; # is that another
::platform::generic? or would we miss something else? etc 
	* let alone, nobody uses in reality [namespace unknown] handler of some
namespace to lazy load things FROM OTHER (NOT CHILD!) NAMESPACES, 
one would either use global unknown, tclIndex, etc.
But if yes, see P.1 

Regarding counter-counterpoint part, I wrote it because someone made an
effort to write a counterpoint directly to the TIP, 
despite the discussion in Tcl-core mailing list was already started and
I also already asked the same questions to the counterparty, 
but got either no or only dubious answers. 

Anyway, I cannot imagine a real-life, consistent and rational example
where the enhancement of [namespace unknown] may bother. 

As for reasonability at time of TIP 181 - well, the time runs and what
was reasonable 20 years ago, may be a bit outdated now.
I can provide a lot of examples with several TIPs that modifying other
TIPs or some previous basic principles in incompatible way, 
and one of the arguments was always - in "newer version" the backwards
compatibility is not a must. Why it can not be applied here
is a riddle for me (let alone TIP 181 was for 8.5, and TIP 689 was
planned for 9.x), but OK. 

Regarding "expando" (don't like the word for lazy load handling) or
rather more regarding my suggestion (new handler at the NS-resolver
level),
I also don't see good chances for success about that (TIP, TCT, etc),
since I have already troubles to convince by simplest things like TIP
689.
(This was also one of the reasons why I tried to start with unknown
handler firstly).
Fortunately, I have no problems with it, as I already implemented it in
my fork and have been using it successfully for almost a dozen years.

Thus, sorry that I took up your time for that. 

Regards,
Serg. 

29.07.2025 18:40, apnmbx-public--- via Tcl-Core wrote: 

> Sergey, 
> 
> You wrote 
> 
>> Why? I mean why it is logical in your opinion that the command ::A::B::C::cmd is resolved.... 
> 
> Where did I say it is logical? I didn't write the counterpoint or counter-counterpoint you are referencing. My point is, rightly or wrongly, the behavior was thought reasonable at the time the TIP was accepted. Some (many?) people still think so. Others (I am one) are wary about unforeseen impact of the change in behavior. The fact that 689 still languishes is proof of that. I am therefore suggesting expando as a way to get past the deadlock and fix the actual use case (dynamic definition of commands in a namespace, whether by loading or otherwise). 
> 
> But since you asked, the following seems a plausible use of current behavior - lazy loading of packages known to be "heavy weight". This is the mirror image of the case you are describing except it is the caller that is doing the lazy load. Assuming the "platform" packages is to be lazy loaded, a namespace that uses it may define a handler to load it on use. 
> 
> % readFile x.tcl 
> 
> namespace eval pkg1 { 
> 
> proc unknownHandler {args} { 
> 
> # No error checking! Infinite loops possible!! 
> 
> package require platform 
> 
> tailcall {*}$args 
> 
> } 
> 
> namespace unknown unknownHandler 
> 
> } 
> 
> % source x.tcl 
> 
> unknownHandler 
> 
> % namespace eval pkg1 {::platform::generic} 
> 
> win32-x86_64 
> 
> It is not clear to me that this would continue to work with 689, in particular if two packages defines such handlers and happen to invoke each other behavior would be changed. Irrespective of whether 689 deals with this or not, the point is I am hesitant to say the current behavior is pointless or unusable. 
> 
> But getting back to the main thrust of my post, if there is disagreement on 689, let us at least move ahead on solving the lazy load / dynamic command definition use case. As to exactly how I cannot say because I do not know the resolver well enough. At first glance, it seems like it would have broader ramification than the more limited "expando" approach, for better or worse. That can be discussed further and worked out. The auto_index_ns approach (as opposed to resolver and expand) I am a bit more skeptical of because it seems to me it implies you know all the names to registered ahead of time which is not the case if for example you were using this facility to bind to expando COM interfaces (twapi does this right now, with TclOO, not namespaces). 
> 
> /Ashok 
> 
> FROM: Dipl. Ing. Sergey G. Brester <se...@us...> 
> SENT: Tuesday, July 29, 2025 3:21 PM
> TO: apn...@ya...
> CC: 'Tcl List Core' <tcl...@li...>
> SUBJECT: Re: [TCLCORE] FW: On namespace unknown and TIP 689 
> 
> Well, I don't understand the necessity to have 2 handlers for the same thing, especially strange because 
> [namespace unknown] for global namespace (default ::unknown) already used for load purposes.
> However I already suggested similar thing like "expando" (already several times, but it did not receive the understanding and support),
> although it was rather an extension for the NS-resolver phase (see below). 
> 
> I already suggested in the TIP to make the new behavior of [namespace unknown] optional (e. g. with some option like -extern or -across or whatever).
> But firstly I'd like to get some answers to questions I asked in counterpoint to counterpoint in the TIP, for instance this: 
> 
> Why? I mean why it is logical in your opinion that the command ::A::B::C::cmd is resolved to be belonging to the namespace ::A::B::C, but the namespace unknown will be invoked in current namespace, no matter what is it? What is the logic behind it?
> And which particular use case shall be covered by this behavior? 
> 
> Neither I ever heard some reasonable arguments in all the discussion before against the TIP (excepting ridiculous "it is wrong because it is wrong"),
> nor the "counterpoint" in the TIP explaining something or makes clear why it may be undesirable (moreover it mixes bananas with cucumbers - namespaces and ensembles),
> nor I could envisage any single use case where the old behavior may be needed or what the new one may then break in the existing code, not by any stretch of imagination.
> Vice versa, many people don't use [namespace unknown] right now, exactly because of the lacks described in TIP 689 (and I have often heard that it is useless). 
> 
> However, I can indeed imagine a completely new facility that could register auto-loading of NS on demand by its access,
> but on resolver phase, so no matter where exactly it needed (command, variable, path, etc).
> In my fork I use array _::auto_index_ns_ to register such NS-loader (similar to _::auto_index_ handling but not for the commands only).
> This could indeed be handled separately and would expect a different TIP.
> 
> The example for that is pretty simple: 
> 
> set ::auto_index_ns(::X::A) {load binary-x-a}
> set ::auto_index_ns(::X::B) {source script-x-b}
> set ::auto_index_ns(::X::C) {namespace eval ::X::C {... large script evaluated on demand ...}}
> ...
> set ::X::A::some-variable; # ::X::A can load here
> ::X::B::some-command ...; # ::X::B can load here
> namespace eval ::Y::C { namespace path {::X::C} }; # ::X::C can load here 
> 
> (in all 3 cases the namespaces would be loaded on demand if they are not yet exists) 
> 
> Alternatively, instead of _::auto_index_ns_ one could make something like _[namespace load name ?loader?]_ to register on-demand loader. 
> 
> Anyway, in opposite to _[namespace unknown]_ it has additional advantages, e. g. NS can be loaded by ANY access to it and one doesn't need to create NS to register the loader, but...
> But I'm pretty sure the [namespace unknown] shall be also fixed (and TIP 689 needs to be accepted) for many reasons. 
> 
> Regards,
> Serg. 
> 
> 29.07.2025 09:35, apnmbx-public--- via Tcl-Core wrote: 
> 
>> Resending as did not seem to go through. 
>> 
>> SENT: Tuesday, July 29, 2025 11:40 AM
>> TO: 'tcl...@li...' <tcl...@li...>
>> SUBJECT: On namespace unknown and TIP 689 
>> 
>> There has been an ongoing debate every few months about the behavior of namespace unknown and TIP 689. Sergey recently also updated a ticket on the matter (if I understood correctly) on benefits in performance through delayed package loading. 
>> 
>> I understand the pros of TIP 689 but am also cognizant of incompatibilities in unforeseen ways, whether real or perceived. 
>> 
>> I wonder if the following proposal would satisfy both parties by allowing dynamic addition of commands to a namespace while greatly reducing (but probably not completely eliminating) the chance of incompatibility. 
>> 
>> A new command is introduced. ("expando" is a placeholder coming from COM terminology) 
>> 
>> namespace eval ns { 
>> 
>> namespace expando expandoProc 
>> 
>> proc expandoProc commandName { 
>> 
>> load or otherwise make commandName available and return 1 
>> 
>> otherwise return 0 
>> 
>> } 
>> 
>> } 
>> 
>> namespace ns2 { 
>> 
>> ::ns::someProc 
>> 
>> } 
>> 
>> The Tcl command resolution is modified as follows. 
>> 
>> <![if !supportLists]>- <![endif]>The command ns::someProc is looked up in the current fashion (namespace path etc.). 
>> 
>> <![if !supportLists]>- <![endif]>If not found, the *TARGET* namespace ns is checked if it has defined an expandoProc. Since it has, it is called. If it returns 1 indicating it created a someProc command in its namespace, the original command is re-executed. 
>> 
>> <![if !supportLists]>- <![endif]>If the ns had not defined an expandoProc or if expandoProc returned 0, the current namespace unknown algorithm is followed. 
>> 
>> With the clock command for example, no preloading needs to be done. The first time clock::format is called, clock's expando proc will load the implementation. 
>> 
>> At the same time, since namespaces explicitly opt-in as expandable, the incompatibilities will be intended and limited. 
>> 
>> Obviously much hand-waving above and details to be sorted out. 
>> 
>> But any obvious (or otherwise) flaws in the above? Thoughts? 
>> 
>> /Ashok 
>> 
>> _______________________________________________
>> 
>> Tcl-Core mailing list
>> 
>> Tcl...@li...
>> 
>> https://lists.sourceforge.net/lists/listinfo/tcl-core [1]
> 
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core [1]
 

Links:
------
[1] https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] Advice requested about support for "-singleproc 0" in the Tk test suite

[Erik L. <el...@xs...>]

On 29/07/2025 08:34, apn...@ya... wrote

> I do find -singleproc useful in the Tcl test suite, both for uncovering bugs
> and for isolation
I expect that these reasons also apply to Tk; I don't see why not.
>   but I'm afraid I don't have enough familiarity with Tk to
> opine.
>
> The commit you referenced in [A] is curious though in that -singleproc is
> not the only command line option that is overridden. Options -testdir and
> -loadfile are also overridden. And the commit comment seems irrelevant.
> Maybe you could remove the -singleproc override and see what errors crop up
> that mandated the override ?

Thanks for the suggestion.

Removing the override in all.tcl, and passing "-singleproc 0" on the 
command line, did not provoke any problem for the Tk test suite.
It ran just fine, using a separate interpreter for each test file.

So it seems that problems for the Tk test suite with "-singleproc 0" 
were not the reason for a forced "singleproc 1".

>   If they are something not fixable, perhaps that
> may be justification for simplification.

I guess you meant the converse: that it would be a justification for 
keeping the current override in all.tcl. ?!

Regards,
Erik Leunissen.

--


> Sorry, can't provide any concrete feedback.
>
> /Ashok
>
> -----Original Message-----
> From: Erik Leunissen via Tcl-Core<tcl...@li...> 
> Sent: Sunday, July 27, 2025 12:42 AM
> To: 'Tcl List Core'<tcl...@li...>
> Subject: [TCLCORE] Advice requested about support for "-singleproc 0" in the
> Tk test suite
>
> L.S.
>
> The tcltest harness provides an option "-singleproc", which enforces that
> each
> test file is evaluated in a separate process (and therefore in a separate
> interpreter) if the option passes the value "0" along.
>
> Currently the Tk test suite overrides any option "-singleproc" om the
> command line
> with "-singleproc 1" (in all.tcl). That makes "-singleproc 0" is
> unsupported,
> intentionally as it seems. For a history of the option "-singleproc" in
> all.tcl,
> please see [A].
>
> While working on the Tk test suite, I see that considerable simplifications
> can
> be made to the initialization of each testfile [B]. But this is only so in
> the
> case of "-singleproc 1".
>
> I would like to implement these simplifications. However, this only makes
> sense
> if support for mode "-singleproc 0" is indeed to be ruled out for the
> foreseeable
> future.
>
> Could TCT-members / maintainers please advise me about that ?
>
> Thanks in advance for your responses,
> Erik Leunissen.
> --
>
> [A] The Tk test suite has always sourced test files into a single common
> interpreter:
>       1. in the very beginning, it didn't specify the option "-singleproc",
> letting
>          the default (-singleproc 1) prevail. This was allowed to be
> overridden by
>          a "-singleproc 0" on the command line.
>
>       2. in 2002 (commit [c483179b915]) the option "singleproc 1" was
> explicitly
>          introduced in "all.tcl", but it was still allowed to be overridden
> by a
>          "-singleproc 0" on the command line:
>
> 	
> [https://core.tcl-lang.org/tk/fdiff?v1=85629ca6e929932d&v2=caa64a2916c7f527]
>
>       3. in 2007 (commit [38bf2cb83c]), using the command line to override
>          "-singleproc 1" was disabled. This made the usage of a separate
> interpreter
>          for each test file unsupported:
>
> 	
> [https://core.tcl-lang.org/tk/fdiff?v1=22b27e50675deec3&v2=5fd3e253a7d904de]
>
> [B]  The most important simplification is the replacement of the
>        -loadfile/loadTestedCommand mechanism. This mechanism, which is meant
> for the
>        mode "-singleproc 0", is somewhat involved by its nature, because it
> needs to
>        cross process boundaries. This mechanism can be replaced with a much
> more simple
>        alternative if we don't need to consider "-singleproc 0". Also, the
> command to
>        load package tcltest can be removed, because that has already been
> done (in
>        all.tcl, in the same process).
> --
>
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>

[TCLCORE] Tk 9.1a0 Release Candidate

[Donald G P. <don...@ni...>]

Now available from

https://sourceforge.net/projects/tcl/files/Tcl/9.1a0/

is the RC2 release candidate of the first alpha release of Tk 9.1.

This is the third of a sequence of candidate releases leading to the release of
Tk 9.1a0. Testing of builds and operations on multiple platforms is invited.  Open
tickets on any problems discovered, or raise the issue in a reply to this message.

It is intended that the RC2 release will become the Tk 9.1a0 release on July 303,
unless some release-blocking flaw is discovered and reported.

Release notes are also available for review and correction.

Thank you for your contributions and assistance.

-- 
| Don Porter            Applied and Computational Mathematics Division |
| don...@ni...             Information Technology Laboratory |
| http://math.nist.gov/~DPorter/                                  NIST |
|______________________________________________________________________|

[TCLCORE] Tcl 9.1a0 Release Candidate

[Donald G P. <don...@ni...>]

Now available from

https://sourceforge.net/projects/tcl/files/Tcl/9.1a0/

is the RC2 release candidate of the first alpha release of Tcl 9.1.

This is the third of a sequence of candidate releases leading to the release of
Tcl 9.1a0. Testing of builds and operations on multiple platforms is invited.  Open
tickets on any problems discovered, or raise the issue in a reply to this message.

It is intended that the RC2 release will become the Tcl 9.1a0 release on July 30,
unless some release-blocking flaw is discovered and reported.

Release notes are also available for review and correction.

Thank you for your contributions and assistance.

[TCLCORE] tcl-parser and tcl-debugger in tcltk-depot

[<apn...@ya...>]

The Tcl parser and Tcl debugger components originating from Scriptics'
TclPro suite are now housed at https://github.com/tcltk-depot/tcl-parser and
https://github.com/tcltk-depot/tcl-debugger respectively.

 

With respect to the latter, changes include

*	Updated to run under Tcl/Tk 9.0 (couple of hundred namespace
variable declarations missing!)
*	Removal of vestigial TclPro components not required by the debugger
*	Refactoring to remove internal dependencies
*	Use of external parser and Tcllib cmdline packages. These need to be
installed before running the debugger.
*	Working test suite

 

I have tried basic functionality on Windows and Ubuntu with 9.0 and 8.6
including remote debug. No doubt more extensive testing would be useful.

 

Known issues: 8.6 starkit functionality is broken. Fixing should be
straightforward (mostly the file manifest)

 

Unknown issues: it is not clear if the debugger ever supported coroutines
and NRE.

 

Maintainers, or at least contributors, would be welcome.

 

/Ashok

 

 

Re: [TCLCORE] FW: On namespace unknown and TIP 689

[<apn...@ya...>]

Sergey,

 

You wrote

 

>Why? I mean why it is logical in your opinion that the command ::A::B::C::cmd is resolved….

 

Where did I say it is logical? I didn’t write the counterpoint or counter-counterpoint you are referencing. My point is, rightly or wrongly, the behavior was thought reasonable at the time the TIP was accepted. Some (many?) people still think so. Others (I am one) are wary about unforeseen impact of the change in behavior. The fact that 689 still languishes is proof of that. I am therefore suggesting expando as a way to get past the deadlock and fix the actual use case (dynamic definition of commands in a namespace, whether by loading or otherwise).

 

But since you asked, the following seems a plausible use of current behavior – lazy loading of packages known to be “heavy weight”. This is the mirror image of the case you are describing except it is the caller that is doing the lazy load. Assuming the “platform” packages is to be lazy loaded, a namespace that uses it may define a handler to load it on use.

 

% readFile x.tcl

namespace eval pkg1 {

    proc unknownHandler {args} {

        # No error checking! Infinite loops possible!!

        package require platform

        tailcall {*}$args

    }

    namespace unknown unknownHandler

}

 

% source x.tcl

unknownHandler

% namespace eval pkg1 {::platform::generic}

win32-x86_64

 

It is not clear to me that this would continue to work with 689, in particular if two packages defines such handlers and happen to invoke each other behavior would be changed. Irrespective of whether 689 deals with this or not, the point is I am hesitant to say the current behavior is pointless or unusable.

 

But getting back to the main thrust of my post, if there is disagreement on 689, let us at least move ahead on solving the lazy load / dynamic command definition use case. As to exactly how I cannot say because I do not know the resolver well enough. At first glance, it seems like it would have broader ramification than the more limited “expando” approach, for better or worse. That can be discussed further and worked out. The auto_index_ns approach (as opposed to resolver and expand) I am a bit more skeptical of because it seems to me it implies you know all the names to registered ahead of time which is not the case if for example you were using this facility to bind to expando COM interfaces (twapi does this right now, with TclOO, not namespaces).

 

/Ashok

 

From: Dipl. Ing. Sergey G. Brester <se...@us...> 
Sent: Tuesday, July 29, 2025 3:21 PM
To: apn...@ya...
Cc: 'Tcl List Core' <tcl...@li...>
Subject: Re: [TCLCORE] FW: On namespace unknown and TIP 689

 

Well, I don't understand the necessity to have 2 handlers for the same thing, especially strange because 
[namespace unknown] for global namespace (default ::unknown) already used for load purposes.
However I already suggested similar thing like "expando" (already several times, but it did not receive the understanding and support),
although it was rather an extension for the NS-resolver phase (see below).

I already suggested in the TIP to make the new behavior of [namespace unknown] optional (e. g. with some option like -extern or -across or whatever).
But firstly I'd like to get some answers to questions I asked in counterpoint to counterpoint in the TIP, for instance this:

Why? I mean why it is logical in your opinion that the command ::A::B::C::cmd is resolved to be belonging to the namespace ::A::B::C, but the namespace unknown will be invoked in current namespace, no matter what is it? What is the logic behind it?
And which particular use case shall be covered by this behavior?

Neither I ever heard some reasonable arguments in all the discussion before against the TIP (excepting ridiculous "it is wrong because it is wrong"),
nor the "counterpoint" in the TIP explaining something or makes clear why it may be undesirable (moreover it mixes bananas with cucumbers - namespaces and ensembles),
nor I could envisage any single use case where the old behavior may be needed or what the new one may then break in the existing code, not by any stretch of imagination.
Vice versa, many people don't use [namespace unknown] right now, exactly because of the lacks described in TIP 689 (and I have often heard that it is useless).

However, I can indeed imagine a completely new facility that could register auto-loading of NS on demand by its access,
but on resolver phase, so no matter where exactly it needed (command, variable, path, etc).
In my fork I use array ::auto_index_ns to register such NS-loader (similar to ::auto_index handling but not for the commands only).
This could indeed be handled separately and would expect a different TIP.

The example for that is pretty simple:

set ::auto_index_ns(::X::A) {load binary-x-a}
set ::auto_index_ns(::X::B) {source script-x-b}
set ::auto_index_ns(::X::C) {namespace eval ::X::C {... large script evaluated on demand ...}}
...
set ::X::A::some-variable;                          # ::X::A can load here
::X::B::some-command ...;                           # ::X::B can load here
namespace eval ::Y::C { namespace path {::X::C} };  # ::X::C can load here

(in all 3 cases the namespaces would be loaded on demand if they are not yet exists)

Alternatively, instead of ::auto_index_ns one could make something like [namespace load name ?loader?] to register on-demand loader.

Anyway, in opposite to [namespace unknown] it has additional advantages, e. g. NS can be loaded by ANY access to it and one doesn't need to create NS to register the loader, but...
But I'm pretty sure the [namespace unknown] shall be also fixed (and TIP 689 needs to be accepted) for many reasons.

Regards,
Serg.

29.07.2025 09:35, apnmbx-public--- via Tcl-Core wrote:

Resending as did not seem to go through.

 


Sent: Tuesday, July 29, 2025 11:40 AM
To: 'tcl...@li...' <tcl...@li... <mailto:tcl...@li...> >
Subject: On namespace unknown and TIP 689

 

There has been an ongoing debate every few months about the behavior of namespace unknown and TIP 689. Sergey recently also updated a ticket on the matter (if I understood correctly) on benefits in performance through delayed package loading.

 

I understand the pros of TIP 689 but am also cognizant of incompatibilities in unforeseen ways, whether real or perceived.

 

I wonder if the following proposal would satisfy both parties by allowing dynamic addition of commands to a namespace while greatly reducing (but probably not completely eliminating) the chance of incompatibility.

 

A new command is introduced. (“expando” is a placeholder coming from COM terminology)

 

namespace eval ns {

    namespace expando expandoProc

    proc expandoProc commandName {

        load or otherwise make commandName available and return 1

        otherwise return 0

    }

}

 

namespace ns2 {

   ::ns::someProc

}

 

The Tcl command resolution is modified as follows. 

<![if !supportLists]>-          <![endif]>The command ns::someProc is looked up in the current fashion (namespace path etc.).

<![if !supportLists]>-          <![endif]>If not found, the *target* namespace ns is checked if it has defined an expandoProc. Since it has, it is called. If it returns 1 indicating it created a someProc command in its namespace, the original command is re-executed. 

<![if !supportLists]>-          <![endif]>If the ns had not defined an expandoProc or if expandoProc returned 0, the current namespace unknown algorithm is followed.

 

With the clock command for example, no preloading needs to be done. The first time clock::format is called, clock’s expando proc will load the implementation.

 

At the same time, since namespaces explicitly opt-in as expandable, the incompatibilities will be intended and limited.

 

Obviously much hand-waving above and details to be sorted out.

 

But any obvious (or otherwise) flaws in the above? Thoughts?

 

/Ashok

 

_______________________________________________
Tcl-Core mailing list
Tcl...@li... <mailto:Tcl...@li...> 
https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] [EXTERNAL] Biweekly telco on28th of July at 12:00 UTC

[Donald G P. <don...@ni...>]

Just a few comments on the meeting notes.

On 7/28/25 08:50, Harald Oehlmann wrote:
> 7.1) TCLPro tools start to be in the orphaned git.
> Ashok works on the debugger.
> Not sure, if the debugger handles coroutines.

FYI, the debugging features of TclX also never got adapted to the new NRE architecture associated with coroutines.

> 7.3) State of TCL for SuSE
> The build of bundled packages for 9.0 and 8.6 automatically at the same time is an issue and is to solve. Proposal is to copy the bundled packages out.

The distribution file tcl-coreM.m.p-src.tar.gz is intended to serve this need.
No need to "copy out" anything.  Just use the release artifact that omits
the tcl/pkgs content.

If this has not been a useful solution, let me know and I'll take it out of the release process.

-- 
| Don Porter            Applied and Computational Mathematics Division |
| don...@ni...             Information Technology Laboratory |
| http://math.nist.gov/~DPorter/                                  NIST |
|______________________________________________________________________|

Re: [TCLCORE] FW: On namespace unknown and TIP 689

[Dipl. I. S. G. B. <se...@us...>]

 

Well, I don't understand the necessity to have 2 handlers for the same
thing, especially strange because 
[namespace unknown] for global namespace (default ::unknown) already
used for load purposes.
However I already suggested similar thing like "expando" (already
several times, but it did not receive the understanding and support),
although it was rather an extension for the NS-resolver phase (see
below). 

I already suggested in the TIP to make the new behavior of [namespace
unknown] optional (e. g. with some option like -extern or -across or
whatever).
But firstly I'd like to get some answers to questions I asked in
counterpoint to counterpoint in the TIP, for instance this: 

Why? I mean why it is logical in your opinion that the command
::A::B::C::cmd is resolved to be belonging to the namespace ::A::B::C,
but the namespace unknown will be invoked in current namespace, no
matter what is it? What is the logic behind it?
And which particular use case shall be covered by this behavior? 

Neither I ever heard some reasonable arguments in all the discussion
before against the TIP (excepting ridiculous "it is wrong because it is
wrong"),
nor the "counterpoint" in the TIP explaining something or makes clear
why it may be undesirable (moreover it mixes bananas with cucumbers -
namespaces and ensembles),
nor I could envisage any single use case where the old behavior may be
needed or what the new one may then break in the existing code, not by
any stretch of imagination.
Vice versa, many people don't use [namespace unknown] right now, exactly
because of the lacks described in TIP 689 (and I have often heard that
it is useless). 

However, I can indeed imagine a completely new facility that could
register auto-loading of NS on demand by its access,
but on resolver phase, so no matter where exactly it needed (command,
variable, path, etc).
In my fork I use array _::auto_index_ns_ to register such NS-loader
(similar to _::auto_index_ handling but not for the commands only).
This could indeed be handled separately and would expect a different
TIP.

The example for that is pretty simple: 

set ::auto_index_ns(::X::A) {load binary-x-a}
set ::auto_index_ns(::X::B) {source script-x-b}
set ::auto_index_ns(::X::C) {namespace eval ::X::C {... large script
evaluated on demand ...}}
...
set ::X::A::some-variable; # ::X::A can load here
::X::B::some-command ...; # ::X::B can load here
namespace eval ::Y::C { namespace path {::X::C} }; # ::X::C can load
here 

(in all 3 cases the namespaces would be loaded on demand if they are not
yet exists) 

Alternatively, instead of _::auto_index_ns_ one could make something
like _[namespace load name ?loader?]_ to register on-demand loader. 

Anyway, in opposite to _[namespace unknown]_ it has additional
advantages, e. g. NS can be loaded by ANY access to it and one doesn't
need to create NS to register the loader, but...
But I'm pretty sure the [namespace unknown] shall be also fixed (and TIP
689 needs to be accepted) for many reasons. 

Regards,
Serg. 

29.07.2025 09:35, apnmbx-public--- via Tcl-Core wrote: 

> Resending as did not seem to go through. 
> 
> SENT: Tuesday, July 29, 2025 11:40 AM
> TO: 'tcl...@li...' <tcl...@li...>
> SUBJECT: On namespace unknown and TIP 689 
> 
> There has been an ongoing debate every few months about the behavior of namespace unknown and TIP 689. Sergey recently also updated a ticket on the matter (if I understood correctly) on benefits in performance through delayed package loading. 
> 
> I understand the pros of TIP 689 but am also cognizant of incompatibilities in unforeseen ways, whether real or perceived. 
> 
> I wonder if the following proposal would satisfy both parties by allowing dynamic addition of commands to a namespace while greatly reducing (but probably not completely eliminating) the chance of incompatibility. 
> 
> A new command is introduced. ("expando" is a placeholder coming from COM terminology) 
> 
> namespace eval ns { 
> 
> namespace expando expandoProc 
> 
> proc expandoProc commandName { 
> 
> load or otherwise make commandName available and return 1 
> 
> otherwise return 0 
> 
> } 
> 
> } 
> 
> namespace ns2 { 
> 
> ::ns::someProc 
> 
> } 
> 
> The Tcl command resolution is modified as follows. 
> 
> <![if !supportLists]>- <![endif]>The command ns::someProc is looked up in the current fashion (namespace path etc.). 
> 
> <![if !supportLists]>- <![endif]>If not found, the *TARGET* namespace ns is checked if it has defined an expandoProc. Since it has, it is called. If it returns 1 indicating it created a someProc command in its namespace, the original command is re-executed. 
> 
> <![if !supportLists]>- <![endif]>If the ns had not defined an expandoProc or if expandoProc returned 0, the current namespace unknown algorithm is followed. 
> 
> With the clock command for example, no preloading needs to be done. The first time clock::format is called, clock's expando proc will load the implementation. 
> 
> At the same time, since namespaces explicitly opt-in as expandable, the incompatibilities will be intended and limited. 
> 
> Obviously much hand-waving above and details to be sorted out. 
> 
> But any obvious (or otherwise) flaws in the above? Thoughts? 
> 
> /Ashok 
> 
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core [1]
 

Links:
------
[1] https://lists.sourceforge.net/lists/listinfo/tcl-core

[TCLCORE] FW: On namespace unknown and TIP 689

[<apn...@ya...>]

Resending as did not seem to go through.

 


Sent: Tuesday, July 29, 2025 11:40 AM
To: 'tcl...@li...' <tcl...@li...>
Subject: On namespace unknown and TIP 689

 

There has been an ongoing debate every few months about the behavior of
namespace unknown and TIP 689. Sergey recently also updated a ticket on the
matter (if I understood correctly) on benefits in performance through
delayed package loading.

 

I understand the pros of TIP 689 but am also cognizant of incompatibilities
in unforeseen ways, whether real or perceived.

 

I wonder if the following proposal would satisfy both parties by allowing
dynamic addition of commands to a namespace while greatly reducing (but
probably not completely eliminating) the chance of incompatibility.

 

A new command is introduced. ("expando" is a placeholder coming from COM
terminology)

 

namespace eval ns {

    namespace expando expandoProc

    proc expandoProc commandName {

        load or otherwise make commandName available and return 1

        otherwise return 0

    }

}

 

namespace ns2 {

   ::ns::someProc

}

 

The Tcl command resolution is modified as follows. 

-          The command ns::someProc is looked up in the current fashion
(namespace path etc.).

-          If not found, the *target* namespace ns is checked if it has
defined an expandoProc. Since it has, it is called. If it returns 1
indicating it created a someProc command in its namespace, the original
command is re-executed. 

-          If the ns had not defined an expandoProc or if expandoProc
returned 0, the current namespace unknown algorithm is followed.

 

With the clock command for example, no preloading needs to be done. The
first time clock::format is called, clock's expando proc will load the
implementation.

 

At the same time, since namespaces explicitly opt-in as expandable, the
incompatibilities will be intended and limited.

 

Obviously much hand-waving above and details to be sorted out.

 

But any obvious (or otherwise) flaws in the above? Thoughts?

 

/Ashok

Re: [TCLCORE] On namespace unknown and TIP 689

[Harald O. <har...@el...>]

Hi Ashok,
great proposal. I never understood why the source namespace unknown 
handler is called. But this could solve the issue.

Thus, "namespace unknown" is not touched.
A new method:
"namespace expando proc"
is added, which:
a) evaluates "proc" in the scope of the namespace
b) returns 0/1 to follow possible "namespace unknown" path.

IMHO, this is fully backward compatible and would solve the issue.

It would be ok for me to change TIP 689 in this way.

As there are apparently use-cases for both variants of unknown, they 
should both exists. Method name "expando" is ok for me.
Same story as with the monotonic clock...

I suppose, the implementation is difficult, specially, if it comes to 
byte compilation...

Thanks for all,
Harald

Am 29.07.2025 um 08:09 schrieb apnmbx-public--- via Tcl-Core:
> There has been an ongoing debate every few months about the behavior of 
> namespace unknown and TIP 689. Sergey recently also updated a ticket on 
> the matter (if I understood correctly) on benefits in performance 
> through delayed package loading.
> 
> I understand the pros of TIP 689 but am also cognizant of 
> incompatibilities in unforeseen ways, whether real or perceived.
> 
> I wonder if the following proposal would satisfy both parties by 
> allowing dynamic addition of commands to a namespace while greatly 
> reducing (but probably not completely eliminating) the chance of 
> incompatibility.
> 
> A new command is introduced. (“expando” is a placeholder coming from COM 
> terminology)
> 
> namespace eval ns {
> 
>      namespace expando expandoProc
> 
>      proc expandoProc commandName {
> 
>          load or otherwise make commandName available and return 1
> 
>          otherwise return 0
> 
>      }
> 
> }
> 
> namespace ns2 {
> 
>     ::ns::someProc
> 
> }
> 
> The Tcl command resolution is modified as follows.
> 
>   * The command ns::someProc is looked up in the current fashion
>     (namespace path etc.).
>   * If not found, the **target** namespace ns is checked if it has
>     defined an expandoProc. Since it has, it is called. If it returns 1
>     indicating it created a someProc command in its namespace, the
>     original command is re-executed.
>   * If the ns had not defined an expandoProc or if expandoProc returned
>     0, the current namespace unknown algorithm is followed.
> 
> With the clock command for example, no preloading needs to be done. The 
> first time clock::format is called, clock’s expando proc will load the 
> implementation.
> 
> At the same time, since namespaces explicitly opt-in as expandable, the 
> incompatibilities will be intended and limited.
> 
> Obviously much hand-waving above and details to be sorted out.
> 
> But any obvious (or otherwise) flaws in the above? Thoughts?
> 
> /Ashok

Re: [TCLCORE] Advice requested about support for "-singleproc 0" in the Tk test suite

[<apn...@ya...>]

I do find -singleproc useful in the Tcl test suite, both for uncovering bugs
and for isolation but I'm afraid I don't have enough familiarity with Tk to
opine.

The commit you referenced in [A] is curious though in that -singleproc is
not the only command line option that is overridden. Options -testdir and
-loadfile are also overridden. And the commit comment seems irrelevant.
Maybe you could remove the -singleproc override and see what errors crop up
that mandated the override ? If they are something not fixable, perhaps that
may be justification for simplification.

Sorry, can't provide any concrete feedback.

/Ashok

-----Original Message-----
From: Erik Leunissen via Tcl-Core <tcl...@li...> 
Sent: Sunday, July 27, 2025 12:42 AM
To: 'Tcl List Core' <tcl...@li...>
Subject: [TCLCORE] Advice requested about support for "-singleproc 0" in the
Tk test suite

L.S.

The tcltest harness provides an option "-singleproc", which enforces that
each
test file is evaluated in a separate process (and therefore in a separate
interpreter) if the option passes the value "0" along.

Currently the Tk test suite overrides any option "-singleproc" om the
command line
with "-singleproc 1" (in all.tcl). That makes "-singleproc 0" is
unsupported,
intentionally as it seems. For a history of the option "-singleproc" in
all.tcl,
please see [A].

While working on the Tk test suite, I see that considerable simplifications
can
be made to the initialization of each testfile [B]. But this is only so in
the
case of "-singleproc 1".

I would like to implement these simplifications. However, this only makes
sense
if support for mode "-singleproc 0" is indeed to be ruled out for the
foreseeable
future.

Could TCT-members / maintainers please advise me about that ?

Thanks in advance for your responses,
Erik Leunissen.
--

[A] The Tk test suite has always sourced test files into a single common
interpreter:
     1. in the very beginning, it didn't specify the option "-singleproc",
letting
        the default (-singleproc 1) prevail. This was allowed to be
overridden by
        a "-singleproc 0" on the command line.

     2. in 2002 (commit [c483179b915]) the option "singleproc 1" was
explicitly
        introduced in "all.tcl", but it was still allowed to be overridden
by a
        "-singleproc 0" on the command line:

	
[https://core.tcl-lang.org/tk/fdiff?v1=85629ca6e929932d&v2=caa64a2916c7f527]

     3. in 2007 (commit [38bf2cb83c]), using the command line to override
        "-singleproc 1" was disabled. This made the usage of a separate
interpreter
        for each test file unsupported:

	
[https://core.tcl-lang.org/tk/fdiff?v1=22b27e50675deec3&v2=5fd3e253a7d904de]

[B]  The most important simplification is the replacement of the
      -loadfile/loadTestedCommand mechanism. This mechanism, which is meant
for the
      mode "-singleproc 0", is somewhat involved by its nature, because it
needs to
      cross process boundaries. This mechanism can be replaced with a much
more simple
      alternative if we don't need to consider "-singleproc 0". Also, the
command to
      load package tcltest can be removed, because that has already been
done (in
      all.tcl, in the same process).
--


_______________________________________________
Tcl-Core mailing list
Tcl...@li...
https://lists.sourceforge.net/lists/listinfo/tcl-core