tcl-core — Tcl Core Team discussions about Tcl development

Re: [TCLCORE] [EXTERNAL] Tcl_RegisterObjType(&tclIntType)

[Jan N. <jan...@gm...>]

Op di 2 sep 2025 om 23:06 schreef Christian Werner:
> would you like to provide a similar snippet for dicts and possibly
> other candidates, too?

Dicts are lists with an even number of arguments, that's all
we can say.

> The fine point was to know in advance, if the Tcl_Obj is a list or not.
But the nice Tcl_ListObjLength() call will make a scalar into a list,
isn't it? And answer the question with TCL_OK. So the scalar will be
suddenly a list. So the question is shadowed by shimmering somehow.

Tcl 9.0 doesn't shimmer to a list any more, when you use
Tcl_ListObjLength() and the Tcl_Obj has another (scalar)
type already. That's the point. Scalars won't be converted
to a list any more.

In Tcl 8.6, yes this kind of shimmering occurs, unfortunately.
So only use this for Tcl 9.0+.

Hope this helps,
        Jan Nijtmans

Re: [TCLCORE] [EXTERNAL] Tcl_RegisterObjType(&tclIntType)

[Christian W. <Chr...@t-...>]

On 09/02/2025 11:17 PM, Christian Werner wrote:
> On 09/02/2025 11:06 PM, Christian Werner wrote:
>> On 09/02/2025 10:44 PM, Jan Nijtmans wrote:
>>
>> Jan, fine, thanks,
>>
>>> Tcl_Size length;
>>> if (Tcl_ListObjLength(NULL, listObj, &length) == TCL_OK) {
>>>     if (length > 1) {
>>>          // Sure, this can be handled as a list
>>>     } else {
>>>          // Either a single element, or a one-element list,
>>>          // Tcl cannot distinguish between that.
>>>     }
>>> } else {
>>>     // Not a list
>>> }
>>
>> would you like to provide a similar snippet for dicts and possibly
>> other candidates, too?
>>
>> And what about to *document* this *prominently* for all those
>> noobs like me, that we shall be never, ever, ever, ever, refer
>> to the holy Tcl_ObjType.
>
> And what lets me now be somehow clueless: Why then is Tcl_GetObjType()
> a public procedure? Is it nowadays deprecated?

Finally, my conclusion is, that your snippet will not resolve the quest.
The fine point was to know in advance, if the Tcl_Obj is a list or not.
But the nice Tcl_ListObjLength() call will make a scalar into a list,
isn't it? And answer the question with TCL_OK. So the scalar will be
suddenly a list. So the question is shadowed by shimmering somehow.

Am I wrong?
Christian

Re: [TCLCORE] Verge of Merge Warning

[Jan N. <jan...@gm...>]

Op di 2 sep 2025 om 23:12 schreef Marc Culler <cul...@gm...>:
>
> This is a warning that I am on the verge of merging the mac_send branch into main.

No objection at all, on the contrary!   I think it's excellent what
you are doing!

Regards,
    Jan Nijtmans

Re: [TCLCORE] [EXTERNAL] Tcl_RegisterObjType(&tclIntType)

[Christian W. <Chr...@t-...>]

On 09/02/2025 11:06 PM, Christian Werner wrote:
> On 09/02/2025 10:44 PM, Jan Nijtmans wrote:
>
> Jan, fine, thanks,
>
>> Tcl_Size length;
>> if (Tcl_ListObjLength(NULL, listObj, &length) == TCL_OK) {
>>     if (length > 1) {
>>          // Sure, this can be handled as a list
>>     } else {
>>          // Either a single element, or a one-element list,
>>          // Tcl cannot distinguish between that.
>>     }
>> } else {
>>     // Not a list
>> }
>
> would you like to provide a similar snippet for dicts and possibly
> other candidates, too?
>
> And what about to *document* this *prominently* for all those
> noobs like me, that we shall be never, ever, ever, ever, refer
> to the holy Tcl_ObjType.

And what lets me now be somehow clueless: Why then is Tcl_GetObjType()
a public procedure? Is it nowadays deprecated?

[TCLCORE] Verge of Merge Warning

[Marc C. <cul...@gm...>]

This is a warning that I am on the verge of merging the mac_send branch
into main.

The branch re-implements the *send* command on macOS to allow sending to an
interpreter running in a different process from the sender.  The receiving
process is required to be owned by the same user as the sending process and
to be running on the same system.

Some key points:
* The code has been reviewed by Kevin Walzer and Steve Landers.
* I am not writing a TIP on the grounds that this is not adding any new
features; it is only implementing documented features which never got
written in the macOS port.  (I more or less followed the plan outlined in
the comments of the old tkMacOSXSend.c file.)
* The implementation uses Tk's existing mechanism for receiving DoScript
AppleEvents to do  the IPC.
* The CI tests pass.  Unfortunately I had to leave the nonPortable
constraint in place for three tests which pass reliably on my computers,
because sending the DoScript AppleEvent times out on the CI runner.

- Marc

Re: [TCLCORE] [EXTERNAL] Tcl_RegisterObjType(&tclIntType)

[Christian W. <Chr...@t-...>]

On 09/02/2025 10:44 PM, Jan Nijtmans wrote:

Jan, fine, thanks,

> Tcl_Size length;
> if (Tcl_ListObjLength(NULL, listObj, &length) == TCL_OK) {
>     if (length > 1) {
>          // Sure, this can be handled as a list
>     } else {
>          // Either a single element, or a one-element list,
>          // Tcl cannot distinguish between that.
>     }
> } else {
>     // Not a list
> }

would you like to provide a similar snippet for dicts and possibly
other candidates, too?

And what about to *document* this *prominently* for all those
noobs like me, that we shall be never, ever, ever, ever, refer
to the holy Tcl_ObjType.

Thank you.
Christian

Re: [TCLCORE] [EXTERNAL] Tcl_RegisterObjType(&tclIntType)

[Jan N. <jan...@gm...>]

Op di 2 sep 2025 om 22:18 schreef Christian Werner:
> So Don, would you like to provide me some alternative in one of my
> extensions, where it is crucial to know if a Tcl_Obj is a list (or not)
> for further processing.

Tcl_Size length;
if (Tcl_ListObjLength(NULL, listObj, &length) == TCL_OK) {
    if (length > 1) {
         // Sure, this can be handled as a list
    } else {
         // Either a single element, or a one-element list,
         // Tcl cannot distinguish between that.
    }
} else {
    // Not a list
}

Hope this helps,
      Jan Nijtmans

Re: [TCLCORE] [EXTERNAL] Tcl_RegisterObjType(&tclIntType)

[Christian W. <Chr...@t-...>]

On 09/02/2025 09:16 PM, Donald G Porter via Tcl-Core wrote:

> ...If there's a sincere belief that this kind of internals violation is truly
> necessary for something, we should bring that into the light and determine what good
> supported and sensible alternatives can provide the functionality better.

So Don, would you like to provide me some alternative in one of my
extensions, where it is crucial to know if a Tcl_Obj is a list (or not)
for further processing.

Please review (search for "listObjType")
  
   https://androwish.org/home/file?name=jni/topcua/topcua.c&ci=tip

and please suggest a better alternative.

Thank you very much in advance,
Christian

Re: [TCLCORE] [EXTERNAL] Tcl_RegisterObjType(&tclIntType)

[Donald G P. <don...@ni...>]

On 8/4/25 10:16, Miguel Bañón wrote:
> So, any reason for not adding Tcl_RegisterObjType(&tclIntType); to TclInitObjSubsystem?
Yes.

If that registration had continued in Tcl 9, then existing callers of Tcl_GetObjType("int")
would be able to retrieve the Tcl internal pointer &tclIntType and would continue using
it to peer into the internal representations of Tcl_Obj structs of that type in the false
belief that they knew what to do with the internals of a Tcl "int" value.

TIP 484 was adopted and it revised the internals of Tcl integer types.  Removing the
Tcl_ObjType registration was done intentionally so that all callers who can get into
trouble by these internal changes are confronted with the issue during their migration
from Tcl 8 to Tcl 9.

A new routine in Tcl 9, Tcl_GetNumberFromObj() [TIP 638] may be a good alternative to the
discouraged internals pokery.

This is just one concrete demonstration of one reason why Tcl_ObjType registration
is a poor design and anyone still making use of Tcl_GetObjType() should do what they
can to stop it.  If there's a sincere belief that this kind of internals violation is truly
necessary for something, we should bring that into the light and determine what good
supported and sensible alternatives can provide the functionality better.

-- 
| Don Porter            Applied and Computational Mathematics Division |
| don...@ni...             Information Technology Laboratory |
| http://math.nist.gov/~DPorter/                                  NIST |
|______________________________________________________________________|

Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType)

[bagnon <ba...@gm...>]

Yep, I’m fully aware of this matter. 

Since Tcl 8 compiled scripts cannot be ensured to be correct/compatible in 9.x interpreters in any case, my suggestion would be to go for 9.x only with the new and future compiler versions, and let the old code remain in 8.x companion, hence my code only addressing 9.0.

> On 2 Sep 2025, at 09:06, apnmbx-public--- via Tcl-Core <tcl...@li...> wrote:
> 
> Miguel,
>  
> Thanks for uploading tclcompiler and tbcload to tcltk-depot. I there is one design decision that needs to be addressed.
>  
> Both the on-disk file format as well as parsing API’s (EmitTclSize etc.) now use Tcl_Size for integer values. This leads to two issues:
>  
> Tcl8 tbc files will not be loadable in Tcl 9 on 64 bit architectures. This might be acceptable if the decision is to stick to the old tclcompiler for Tcl 8 and the new one for Tcl 9
> More important, scripts compiled by 32-bit Tcl builds will not be loadable by 64-bit Tcl builds. This I think is a more serious limitation.
>  
> Possible solutions:
> Limit byte coded file formats to 32-bit only. I *think* this is acceptable because the byte code engine itself cannot handle 64-bit offsets, argument counts or 64-bit immediate values in the bytecode stream (Donal, correct me if I’m wrong). 64-bit integer values are passed as Tcl_Obj which is not a problem. This will maintain compatibility with Tcl 8 as well as between 32-bit and 64-bit Tcl 9.
> Make byte coded file formats 64-bit. The 32-bit Tcl tclcompile/tbcload version will need to be modified to read 64-bit integers always, and not Tcl_Size. This makes Tcl 9 32- and 64-bit builds compatible but loses Tcl 8 compatibility.
>  
> I do not myself use tclcompiler and no plans to use in the future so it would be nice if folks who use tclcompiler to comment on the preferable approach.
>  
> /Ashok
>  
> From: Miguel Bañón <ba...@gm... <mailto:ba...@gm...>> 
> Sent: Monday, August 4, 2025 9:59 PM
> To: Donal Fellows <don...@ma... <mailto:don...@ma...>>
> Cc: tcl...@li... <mailto:tcl...@li...>
> Subject: Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType)
>  
> Thank you both for your answers. 
>  
> I got tdbcload from https://wiki.tcl-lang.org/page/Porting+extensions+to+Tcl+9 <https://wiki.tcl-lang.org/page/Porting+extensions+to+Tcl+9>
>  
> https://www.tcl3d.org/bawt/download/InputLibs/tbcload-1.7.2.7z <https://www.tcl3d.org/bawt/download/InputLibs/tbcload-1.7.2.7z> claims to be ported, but fails to load because of this issue
>  
> Is there any other source for a 9-ready tbcload?
>  
> Best regards,
>  
> Miguel
>  
> 
> 
>> On Aug 4, 2025, at 5:50 PM, Donal Fellows <don...@ma... <mailto:don...@ma...>> wrote:
>>  
>> A bigger practical change for tbcload in 9.1 is that there's a new aux data type (for the new kind of jump table index type); those require extra handling. The changes for 8.6 to 9.0 should be trivial. There may be new opcodes (I forget if this is so) but they should be trivial extensions; the semantic differences don't affect the ability of the engine to serialise. (A rebuild is still necessary, but should be trivial.)
>>  
>> 9.1 currently is expected to be able to read bytecode created with 9.0, but the reverse is definitely not true.
>>  
>> Donal.
>>  
>> -------- Original message --------
>> From: Harald Oehlmann <har...@el... <mailto:har...@el...>> 
>> Date: 04/08/2025 15:26 (GMT+00:00) 
>> To: Miguel Bañón <ba...@gm... <mailto:ba...@gm...>> 
>> Cc: tcl...@li... <mailto:tcl...@li...>
>> Subject: Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType) 
>>  
>> AFAIK there is already a 9.0 ready tdbcload. I don't remember where.
>> Remark, that 9.1 will be considerably different, as there are a lot of 
>> new byte codes.
>> Andreas reported about this in one of the biweekly meetings.
>> 
>> 
> 
>  
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li... <mailto:Tcl...@li...>
> https://lists.sourceforge.net/lists/listinfo/tcl-core <https://lists.sourceforge.net/lists/listinfo/tcl-core>

Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType)

[Donal F. <don...@ma...>]

There are no 64-bit immediate values in the bytecode, and I have no plans to add them; non-32-bit values are significantly rarer in user code. 2GB of bytecode would be a lot, so internal offsets and indices aren't a big hurry either. The only real concern would be very large string literals, and those aren't coming up yet.

I wouldn't really expect saved 8.* bytecode to work with 9.*; it might, but it definitely isn't a promise we ought to make and isn't a design goal. The 9.0 to 9.1 transition is a bigger issue. There are a number of bytecode opcodes that are now deprecated (in favour of other ops, mostly moving from single-byte arguments to 4-byte arguments) as well as several new opcodes (and a new auxiliary data type).

At some point, we ought to revisit the serialisation and deserialisation, to have a think about what bits should be in Tcl itself...

Donal

-------- Original message --------
From: apnmbx-public--- via Tcl-Core <tcl...@li...>
Date: 02/09/2025 08:07 (GMT+00:00)
To: 'Miguel Bañón' <ba...@gm...>
Cc: tcl...@li...
Subject: Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType)



  *   Tcl8 tbc files will not be loadable in Tcl 9 on 64 bit architectures. This might be acceptable if the decision is to stick to the old tclcompiler for Tcl 8 and the new one for Tcl 9
  *   More important, scripts compiled by 32-bit Tcl builds will not be loadable by 64-bit Tcl builds. This I think is a more serious limitation.
 Possible solutions:

  *   Limit byte coded file formats to 32-bit only. I *think* this is acceptable because the byte code engine itself cannot handle 64-bit offsets, argument counts or 64-bit immediate values in the bytecode stream (Donal, correct me if I’m wrong). 64-bit integer values are passed as Tcl_Obj which is not a problem. This will maintain compatibility with Tcl 8 as well as between 32-bit and 64-bit Tcl 9.

Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType)

[<apn...@ya...>]

Miguel,

 

Thanks for uploading tclcompiler and tbcload to tcltk-depot. I there is one design decision that needs to be addressed.

 

Both the on-disk file format as well as parsing API’s (EmitTclSize etc.) now use Tcl_Size for integer values. This leads to two issues:

 

*	Tcl8 tbc files will not be loadable in Tcl 9 on 64 bit architectures. This might be acceptable if the decision is to stick to the old tclcompiler for Tcl 8 and the new one for Tcl 9
*	More important, scripts compiled by 32-bit Tcl builds will not be loadable by 64-bit Tcl builds. This I think is a more serious limitation.

 

Possible solutions:

*	Limit byte coded file formats to 32-bit only. I *think* this is acceptable because the byte code engine itself cannot handle 64-bit offsets, argument counts or 64-bit immediate values in the bytecode stream (Donal, correct me if I’m wrong). 64-bit integer values are passed as Tcl_Obj which is not a problem. This will maintain compatibility with Tcl 8 as well as between 32-bit and 64-bit Tcl 9.
*	Make byte coded file formats 64-bit. The 32-bit Tcl tclcompile/tbcload version will need to be modified to read 64-bit integers always, and not Tcl_Size. This makes Tcl 9 32- and 64-bit builds compatible but loses Tcl 8 compatibility.

 

I do not myself use tclcompiler and no plans to use in the future so it would be nice if folks who use tclcompiler to comment on the preferable approach.

 

/Ashok

 

From: Miguel Bañón <ba...@gm...> 
Sent: Monday, August 4, 2025 9:59 PM
To: Donal Fellows <don...@ma...>
Cc: tcl...@li...
Subject: Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType)

 

Thank you both for your answers. 

 

I got tdbcload from https://wiki.tcl-lang.org/page/Porting+extensions+to+Tcl+9

 

https://www.tcl3d.org/bawt/download/InputLibs/tbcload-1.7.2.7z claims to be ported, but fails to load because of this issue

 

Is there any other source for a 9-ready tbcload?

 

Best regards,

 

Miguel

 





On Aug 4, 2025, at 5:50 PM, Donal Fellows <don...@ma... <mailto:don...@ma...> > wrote:

 

A bigger practical change for tbcload in 9.1 is that there's a new aux data type (for the new kind of jump table index type); those require extra handling. The changes for 8.6 to 9.0 should be trivial. There may be new opcodes (I forget if this is so) but they should be trivial extensions; the semantic differences don't affect the ability of the engine to serialise. (A rebuild is still necessary, but should be trivial.)

 

9.1 currently is expected to be able to read bytecode created with 9.0, but the reverse is definitely not true.

 

Donal.

 

-------- Original message --------

From: Harald Oehlmann <har...@el... <mailto:har...@el...> > 

Date: 04/08/2025 15:26 (GMT+00:00) 

To: Miguel Bañón <ba...@gm... <mailto:ba...@gm...> > 

Cc: tcl...@li... <mailto:tcl...@li...>  

Subject: Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType) 

 

AFAIK there is already a 9.0 ready tdbcload. I don't remember where.
Remark, that 9.1 will be considerably different, as there are a lot of 
new byte codes.
Andreas reported about this in one of the biweekly meetings.



 

Re: [TCLCORE] valgrind exceptions

[<apn...@ya...>]

Marc,

 

(CC’ing tcl-core as it is a broader discussion and others may have suggestions)

 

The issue is not the Tcl_Free() one line call, it is more about *when* that call should be made. There are multiple threads sharing that initialized value, including potentially threads that do not create interpreters at all. There are at least two things to be taken care of before freeing

 

*	ensuring all other threads have exited
*	ensuring the *current* thread, the last one exiting, that frees the value does not subsequently try and access the file system as part of some other clean up after the value is freed, further complicated in this particular case because zipfs has an incestuous relationship with the native file system

 

The first requires non-trivial machinery with locks and reference counting; the latter requires careful review and even then, given past experiences with Tcl finalization, carries risk. Both are doable but only increases bloat for questionable gain. Note that in practice, Tcl does not do finalization at all, unless a specific environment variable is set. So the value of all that machinery is doubtful.

 

Tcl actually does have a process global value mechanism to help with the first problem. Aside from it not solving the second, there are two other considerations that gave me pause. The first, perhaps minor, is the overhead of locking, Tcl_Obj conversion, TSD access for accessing what is essentially a constant value once initialized. The second, specific to this case, is that I would prefer to have the value have the exact byte sequence that GetModuleName or dladdr returned to me without going through the system encoding roundtripping done by the global process value code which is not always an identity transform.

 

Of course, these exceptions should not be made in instances where multiple allocations are possible because those are genuine memory leaks but for these “pseudo-static” cases, I think it is a more robust solution.

 

Alternative could be a static array of MAX_PATH / PATH_MAX size though that would require additional locking.

 

BTW, there is already a valgrind-suppress file in use, so it is a question of adding an entry to it.

 

But we’ll wait if folks have comments or other suggestions.

 

/Ashok

 

 

 

From: Marc Culler <cul...@gm...> 
Sent: Monday, September 1, 2025 7:50 PM
To: apn...@ya...
Subject: Re: [TCLCORE] valgrind exceptions

 

I just added a similar char pointer, for a path which only gets set once ,in the mac_send branch.  I didn't know valgrind would complain if I did not free it.  But, given that it does complain, I think it is better to just free it as part of the cleanup.  Surely it is less complicated to add one line of code than to create a valgrind exception.  (I know how to add a ckfree command; I have no idea how to create a valgrind exception.)  This is such a common situation that I think following this path will lead to many, many valgrind exceptions and I think it will be simpler in the long run, and cleaner, to just free those variables.

 

- Marc

 

On Mon, Sep 1, 2025 at 12:40 AM apnmbx-public--- via Tcl-Core <tcl...@li... <mailto:tcl...@li...> > wrote:

In a recent check-in related to zipfs initialization, I allocate a string holding a path. This is global, initialized once, shared by threads. I have not bothered to deallocate it as I do not see it worth the added machinery required to delete at runtime (you have to be careful all threads exited etc.).

 

Valgrind of course shows a leak of a 100 bytes or so.

 

Anybody object to adding a valgrind exception for such cases?

 

/Ashok

_______________________________________________
Tcl-Core mailing list
Tcl...@li... <mailto:Tcl...@li...> 
https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] Oracle data base drivers

[Donal F. <don...@ma...>]

It was probably not part of the TDBC baseline because it depended on having the Oracle client library present, which is very much a blocker for many users. I suppose trickery could be done with soft loading the shared library version, but that wasn't done (likely because of workload).

Sometimes the explanation is very simple. :-D

Donal.

-------- Original message --------
From: Harald Oehlmann <har...@el...>
Date: 01/09/2025 16:28 (GMT+00:00)
To: Tcl Core List <tcl...@li...>
Subject: [TCLCORE] Oracle data base drivers

- why is the tdbc driver not part of the tdbc distribution?

[TCLCORE] Oracle data base drivers

[Harald O. <har...@el...>]

Miguel Bañón reported just about the port of the oratcl driver to TCL 9.0:
https://github.com/bagnongithub/oratcl

Here is a tdbc driver for Oracle data base:
https://fossil.sowaswie.de/tdbc_oratcl/home

There are two questions:
- is oratcl a good candidate for the tcl depot?
- why is the tdbc driver not part of the tdbc distribution?

Thanks for any comments,
Harald

Re: [TCLCORE] Need help with Tcl-GLib event integration

[Harald O. <har...@el...>]

Kevin,
I just want to express my appreciation that you follow this tedious but 
helpful feature with so much energy. I can imagine, that you feel a bit 
alone sometimes. Please feel supported and come back in any cases, even 
if it feels stupid. The common goal is better Tk.

And thanks for sharing your chatgbt vs wiki challenge. As it was in 
chess human against machine, one day, chatgpt will win, but we are not 
there jet.

Thanks for all,
Harald

Am 01.09.2025 um 00:03 schrieb Kevin Walzer:
> A quick follow up...
> 
> As it turns out, my C code was plenty robust - in fact, I threw out a 
> lot of the modifications suggested by AI and went with a simple custom 
> event source as suggested at the Tcl wiki. There's even an example there 
> of Gtk event loop integration; it was trivial to swap the Gtk calls for 
> GLib.

[TCLCORE] valgrind exceptions

[<apn...@ya...>]

In a recent check-in related to zipfs initialization, I allocate a string
holding a path. This is global, initialized once, shared by threads. I have
not bothered to deallocate it as I do not see it worth the added machinery
required to delete at runtime (you have to be careful all threads exited
etc.).

 

Valgrind of course shows a leak of a 100 bytes or so.

 

Anybody object to adding a valgrind exception for such cases?

 

/Ashok

Re: [TCLCORE] CFV: TIP 726

[<apn...@ya...>]

Not looked in detail. Rushing to finish off a few things before I leave for a couple of weeks on unexpected travel with sporadic connectivity, but ...

I agree with changing 0x323c0 -> 0x110000. I had noticed that but did not know the reason for picking the last assigned character as opposed to last valid code point so left it alone.

Regarding invalid code points, I do not have strong opinions and would not object to any changes. As Harald commented in one of the tickets Tcl does not check for validity for strings passed through the C API and it should be up to the application or extension to ensure only valid data comes in (I think we already do this at the script level except possibly for surrogates). Tcl currently may interpret these as Cp1252, replace with U+FFFD, or leave as is depending on the API and specific code point. Garbage in, garbage out... I would prefer Tcl be consistent in handling but other than that any changes should be viewed as something applications should not have relied on anyways (invalid data is undefined behavior).



-----Original Message-----
From: Jan Nijtmans <jan...@gm...> 
Sent: Sunday, August 31, 2025 3:11 PM
To: apn...@ya...
Cc: Tcl Core List <tcl...@li...>
Subject: Re: [TCLCORE] CFV: TIP 726

Op za 30 aug 2025 om 06:43 schreef Ashok:
> Regarding (1) -
>
> As a matter of principle, I prefer to check validity of data before being passed to external libraries (I treat utf8proc as such). When evaluating various libraries, some, like the very fast SIMD based library, expect valid data, others don't, and some differ in their treatment depending on function being called (including utf8proc). I don't think we want to be in the business of revisiting the Tcl implementation on a library change.
>
> But aside from those general principles, there are specific 9.0 compatibility issues with your implementation that I see looking at the code. For example, 9.0 Tcl_UniCharToUpper (and similar) map 0xFFFFFF to 0x1FFFFF while your code seemingly maps it to 0xFFFFFF. On a tangential note, utf8proc definitions of character classifications are not exactly the same as Tcl's historical classifications (including 9.0). I happen to think utf8proc is more appropriate and in line with the Unicode standard but as stated before, TIP 726 strives for full compatibility with Tcl 9.0. Any changes to character classification would be a compatibility change (no matter how small) and require a separate TIP.

You have a point here. The Tcl core only calls those
Tcl_UniCharToXXX() functions with values <= 0x10FFFF
(since that's the maximum number that Tcl_UtfToUnichar() can produce).
There are 3 ranges to consider
closer:
  1)  0xD800 - 0xDFFF.    Since Tcl 8.6 outputs the same value as
input, for compatibility
        we want 9.1 to do the same. It does.
  2) 0x110000 and 0x1FFFFF. Personally, I don't mind much what
Tcl_UniCharToXXX()
        does for values between 0x110000 and 0x1FFFFF. But since 9.0 outputs the
        same value as input, it makes sense to keep doing this in 9.1.
If utf8proc
        does something different (like returning -1), yes we should do
a range check.
   3) Values above 0xFFFFF. There are currently no testcases for that
(those should
       be added, but that's not TIP #726's fault).

Just one more remark for now. In tclUtf.c, I see:
    #define UNICODE_OUT_OF_RANGE(ch) (((ch) & 0x1FFFFF) >= 0x323C0)
Why 0x323C0? In Tcl 8.6 and 9.0, this number was generated from
the UnicodeData.txt file. It was simply the last character present
in the table. It is dangerous to keep this number: What if a future Unicode
version has more characters than that in the 3th plane (or adds a 4th plane)?

So I suggest to change this value to 0x110000 (or 0x40000, with the
remark that this should be increased if the 4th plane gets any characters)

Hope this helps,
    Jan Nijtmans

Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType)

[<apn...@ya...>]

Sorry, ignore my question below. Had missed your earlier post about 9.0.2.

 

From: apnmbx-public--- via Tcl-Core <tcl...@li...> 
Sent: Monday, September 1, 2025 10:45 AM
To: 'Miguel Bañón' <ba...@gm...>; 'Donal Fellows' <don...@ma...>
Cc: tcl...@li...
Subject: Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType)

 

Miguel,

 

Just to clarify as to current state, does this mean the tbcload committed to tcltk-depot still needs work to support 9.0? Or is the failure to load specific to 9.1?

 

/Ashok

 

From: Miguel Bañón <ba...@gm... <mailto:ba...@gm...> > 
Sent: Monday, August 4, 2025 9:59 PM
To: Donal Fellows <don...@ma... <mailto:don...@ma...> >
Cc: tcl...@li... <mailto:tcl...@li...> 
Subject: Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType)

 

Thank you both for your answers. 

 

I got tdbcload from https://wiki.tcl-lang.org/page/Porting+extensions+to+Tcl+9

 

https://www.tcl3d.org/bawt/download/InputLibs/tbcload-1.7.2.7z claims to be ported, but fails to load because of this issue

 

Is there any other source for a 9-ready tbcload?

 

Best regards,

 

Miguel

 

 

On Aug 4, 2025, at 5:50 PM, Donal Fellows <don...@ma... <mailto:don...@ma...> > wrote:

 

A bigger practical change for tbcload in 9.1 is that there's a new aux data type (for the new kind of jump table index type); those require extra handling. The changes for 8.6 to 9.0 should be trivial. There may be new opcodes (I forget if this is so) but they should be trivial extensions; the semantic differences don't affect the ability of the engine to serialise. (A rebuild is still necessary, but should be trivial.)

 

9.1 currently is expected to be able to read bytecode created with 9.0, but the reverse is definitely not true.

 

Donal.

 

-------- Original message --------

From: Harald Oehlmann <har...@el... <mailto:har...@el...> > 

Date: 04/08/2025 15:26 (GMT+00:00) 

To: Miguel Bañón <ba...@gm... <mailto:ba...@gm...> > 

Cc: tcl...@li... <mailto:tcl...@li...>  

Subject: Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType) 

 

AFAIK there is already a 9.0 ready tdbcload. I don't remember where.
Remark, that 9.1 will be considerably different, as there are a lot of 
new byte codes.
Andreas reported about this in one of the biweekly meetings.

 

Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType)

[<apn...@ya...>]

Miguel,

 

Just to clarify as to current state, does this mean the tbcload committed to tcltk-depot still needs work to support 9.0? Or is the failure to load specific to 9.1?

 

/Ashok

 

From: Miguel Bañón <ba...@gm...> 
Sent: Monday, August 4, 2025 9:59 PM
To: Donal Fellows <don...@ma...>
Cc: tcl...@li...
Subject: Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType)

 

Thank you both for your answers. 

 

I got tdbcload from https://wiki.tcl-lang.org/page/Porting+extensions+to+Tcl+9

 

https://www.tcl3d.org/bawt/download/InputLibs/tbcload-1.7.2.7z claims to be ported, but fails to load because of this issue

 

Is there any other source for a 9-ready tbcload?

 

Best regards,

 

Miguel

 





On Aug 4, 2025, at 5:50 PM, Donal Fellows <don...@ma... <mailto:don...@ma...> > wrote:

 

A bigger practical change for tbcload in 9.1 is that there's a new aux data type (for the new kind of jump table index type); those require extra handling. The changes for 8.6 to 9.0 should be trivial. There may be new opcodes (I forget if this is so) but they should be trivial extensions; the semantic differences don't affect the ability of the engine to serialise. (A rebuild is still necessary, but should be trivial.)

 

9.1 currently is expected to be able to read bytecode created with 9.0, but the reverse is definitely not true.

 

Donal.

 

-------- Original message --------

From: Harald Oehlmann <har...@el... <mailto:har...@el...> > 

Date: 04/08/2025 15:26 (GMT+00:00) 

To: Miguel Bañón <ba...@gm... <mailto:ba...@gm...> > 

Cc: tcl...@li... <mailto:tcl...@li...>  

Subject: Re: [TCLCORE] Tcl_RegisterObjType(&tclIntType) 

 

AFAIK there is already a 9.0 ready tdbcload. I don't remember where.
Remark, that 9.1 will be considerably different, as there are a lot of 
new byte codes.
Andreas reported about this in one of the biweekly meetings.



 

Re: [TCLCORE] Need help with Tcl-GLib event integration

[Kevin W. <kw...@co...>]

A quick follow up...

As it turns out, my C code was plenty robust - in fact, I threw out a 
lot of the modifications suggested by AI and went with a simple custom 
event source as suggested at the Tcl wiki. There's even an example there 
of Gtk event loop integration; it was trivial to swap the Gtk calls for 
GLib.

Additional debugging showed that the event flood was happening on focus 
events. After turning over every stone in the C code, I checked the 
script-level implementation and found a {bind all <FocusIn> 
::blah::blah::} that was intended for Windows. Turns out that was not 
only unnecessary on X11, it was downright destructive. Limiting that 
call to Windows seems to have fixed the problem.

This has been holding me up for weeks, and it turns out it's one line of 
code at the script level! Never underestimate Tcl/Tk's power. :-)

I have more polishing and fine-tuning to do on the X11 accessibility 
code, but now the foundation is solid. I'll be able to ask for review 
and a TIP in the near future.

Brian, thanks for your input! It got me thinking about this in different 
ways and I was able to find a solution.

All best,

Kevin

On 8/27/25 7:32 AM, Kevin Walzer wrote:
> Hi Brian,
>
> Thank you. I will take a closer look at your work here. Much appreciated!
>
> —Kevin
>
>> On Aug 27, 2025, at 1:03 AM, Brian Griffin <bri...@ea...> 
>> wrote:
>>
>>  Hi Kevin,
>>
>> I don't know anything about GLib event loop, but I did learn 
>> something important when integrating the Qt event loop with Tcl. The 
>> lesson learned is that there can only be one event loop master. 
>> Everything else is just responding to the events when the master says 
>> so.
>> Qt was interesting because it doesn't really have main loop per se, 
>> but it still needed to know about *all* events. So, I put Tcl as the 
>> top "loop"er (i.e. DoOneEvent). Then translated all Tcl events into 
>> Qt events. Qt would handle it's own events directly, and the "tcl" 
>> events were Qt events, but the callbacks went to Tcl for actual handling.
>> I ended up here because any other way lead to a hang or ignored events.
>>
>> Only one master loop!
>>
>> (Note: my work was before the switch to the new (epoll?) event loop 
>> system.)
>>
>> I hope this gives you some insight.
>> -Brian
>>
>>> On Aug 26, 2025, at 20:09, Kevin Walzer <kw...@co...> wrote:
>>>
>>> Hi all,
>>>
>>> As you may be aware, for the past year I’ve been working on a 
>>> project to introduce accessibility / screen reader support to Tk. 
>>> I’m almost done, but there’s one obstacle I can’t seem to work 
>>> through - integrating the Tcl and GLib event loops on Linux. This is 
>>> required because the ATK accessibility API communicates with 
>>> accessibility clients via the at-spi protocol, which is controlled 
>>> by GLib.
>>>
>>> Put simply, Wish hangs. My latest commits take a bit longer, but 
>>> eventually Wish freezes.
>>>
>>> Looking at the example of the gnocl project, I’ve created a custom 
>>> Tcl event source that consumes events from GLib. It seems 
>>> straightforward but is anything but.
>>>
>>> Here is my current code, based on a suggestion from one of the many 
>>> AI assistants I’ve consulted:
>>> |/* Configure event loop. */ static void Atk_Event_Setup(ClientData 
>>> clientData, int flags) { (void)clientData; static Tcl_Time 
>>> block_time; if (!(flags & TCL_WINDOW_EVENTS)) { return; } /* Ask 
>>> GLib how long it wants to sleep. */ gint timeout = 
>>> g_main_context_iteration(NULL, FALSE); if (timeout > 0) { 
>>> block_time.sec = timeout / 1000; block_time.usec = (timeout % 1000) 
>>> * 1000; } else { /* If GLib has pending events, don't block. */ 
>>> block_time.sec = 0; block_time.usec = 0; } 
>>> Tcl_SetMaxBlockTime(&block_time); } /* Check event queue. */ static 
>>> void Atk_Event_Check(ClientData clientData, int flags) { 
>>> (void)clientData; (void)flags; /* No-op. The real work is done in 
>>> Atk_Event_Setup. */ } |
>>> |Tcl_CreateEventSource (Atk_Event_Setup, Atk_Event_Check, 0); 
>>> Tcl_SetServiceMode (TCL_SERVICE_ALL);|
>>> I’d be grateful for any suggestions.
>>> Thanks,
>>> Kevin
>>> _______________________________________________
>>> Tcl-Core mailing list
>>> Tcl...@li...
>>> https://lists.sourceforge.net/lists/listinfo/tcl-core
>>
>
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] CFV: TIP 726

[Jan N. <jan...@gm...>]

Op za 30 aug 2025 om 06:43 schreef Ashok:
> Regarding (1) -
>
> As a matter of principle, I prefer to check validity of data before being passed to external libraries (I treat utf8proc as such). When evaluating various libraries, some, like the very fast SIMD based library, expect valid data, others don't, and some differ in their treatment depending on function being called (including utf8proc). I don't think we want to be in the business of revisiting the Tcl implementation on a library change.
>
> But aside from those general principles, there are specific 9.0 compatibility issues with your implementation that I see looking at the code. For example, 9.0 Tcl_UniCharToUpper (and similar) map 0xFFFFFF to 0x1FFFFF while your code seemingly maps it to 0xFFFFFF. On a tangential note, utf8proc definitions of character classifications are not exactly the same as Tcl's historical classifications (including 9.0). I happen to think utf8proc is more appropriate and in line with the Unicode standard but as stated before, TIP 726 strives for full compatibility with Tcl 9.0. Any changes to character classification would be a compatibility change (no matter how small) and require a separate TIP.

You have a point here. The Tcl core only calls those
Tcl_UniCharToXXX() functions with values <= 0x10FFFF
(since that's the maximum number that Tcl_UtfToUnichar() can produce).
There are 3 ranges to consider
closer:
  1)  0xD800 - 0xDFFF.    Since Tcl 8.6 outputs the same value as
input, for compatibility
        we want 9.1 to do the same. It does.
  2) 0x110000 and 0x1FFFFF. Personally, I don't mind much what
Tcl_UniCharToXXX()
        does for values between 0x110000 and 0x1FFFFF. But since 9.0 outputs the
        same value as input, it makes sense to keep doing this in 9.1.
If utf8proc
        does something different (like returning -1), yes we should do
a range check.
   3) Values above 0xFFFFF. There are currently no testcases for that
(those should
       be added, but that's not TIP #726's fault).

Just one more remark for now. In tclUtf.c, I see:
    #define UNICODE_OUT_OF_RANGE(ch) (((ch) & 0x1FFFFF) >= 0x323C0)
Why 0x323C0? In Tcl 8.6 and 9.0, this number was generated from
the UnicodeData.txt file. It was simply the last character present
in the table. It is dangerous to keep this number: What if a future Unicode
version has more characters than that in the 3th plane (or adds a 4th plane)?

So I suggest to change this value to 0x110000 (or 0x40000, with the
remark that this should be increased if the 4th plane gets any characters)

Hope this helps,
    Jan Nijtmans

Re: [TCLCORE] CFV: TIP 726

[<apn...@ya...>]

And to one more point (yeah, I know I said this is not a productive use of my time!) ...

 

> I object to making such a change in existing code,

 

I didn’t just randomly make changes in existing code. I was modifying those lines to call the utf8proc functions and used the expression forms that I saw as natural.

 

Now really ‘nuff said from my side.

 

/Ashok

 

-----Original Message-----
From: Jan Nijtmans <jan...@gm...> 
Sent: Saturday, August 30, 2025 8:14 PM
To: apn...@ya...
Cc: Tcl Core List <tcl...@li...>
Subject: Re: [TCLCORE] CFV: TIP 726

 

Op za 30 aug 2025 om 06:43 schreef Ashok:

> Regarding (1) -

 

I'll come back later on this one.

 

> Regarding (2) -

> 

> It is not clear what you mean by more efficient when the compiled instruction stream is identical at least for clang x64 and gcc arm64. See  <https://godbolt.org/z/8afa54eh3> https://godbolt.org/z/8afa54eh3.  I wrote in the style I did because (in my mind of course) it was more natural and clearer to look up the bit in the mask than shifting the mask to look at the bit. I verified with only two compilers and architectures but see no reason to claim either shift would be more efficient. Compilers are smarter than me.

 

More efficient means more efficient, so less instructions. Try the

same experiment

without the "-O2", an you will see that the non-optimized version is longer than

the optimized one. The optimizer is smart enough to realize that a right-shift

is better here, so it changes the left shift to a right-shift.

The original author of the code in tclUtf.c was smart enough

to realize this, and I realize it too. We don't want less efficient code in

debug mode (in which the optimizer is disabled).

 

I object to making such a change in existing code, without

realizing what was behind it. You could have asked before.

 

Sorry,

       Jan Nijtmans

Re: [TCLCORE] CFV: TIP 726

[<apn...@ya...>]

> We don't want less efficient code in debug mode (in which the optimizer is disabled).

 

So if I understand you correctly, the generated code in a release build is identical but your objection is that the debug builds are not?

 

This is just absurd. Sorry.

 

Turning off optimization generates so much rubbish code (from an efficiency point of view) that a couple of instructions are completely immaterial. Worrying about speed in a non-optimized build is something I have never ever heard of and that too in interpreters that are never really speed demons in any case.

 

Being smart is not about bit twiddling any more. That time passed with the compilers that arrived about the turn of the century or before.

 

Do as you please now that the code has been merged into the trunk. Revert the shifts, or not, whatever. Not a productive use of my time to go back and forth on this.

 

/Ashok

 

 

-----Original Message-----
From: Jan Nijtmans <jan...@gm...> 
Sent: Saturday, August 30, 2025 8:14 PM
To: apn...@ya...
Cc: Tcl Core List <tcl...@li...>
Subject: Re: [TCLCORE] CFV: TIP 726

 

Op za 30 aug 2025 om 06:43 schreef Ashok:

> Regarding (1) -

 

I'll come back later on this one.

 

> Regarding (2) -

> 

> It is not clear what you mean by more efficient when the compiled instruction stream is identical at least for clang x64 and gcc arm64. See  <https://godbolt.org/z/8afa54eh3> https://godbolt.org/z/8afa54eh3.  I wrote in the style I did because (in my mind of course) it was more natural and clearer to look up the bit in the mask than shifting the mask to look at the bit. I verified with only two compilers and architectures but see no reason to claim either shift would be more efficient. Compilers are smarter than me.

 

More efficient means more efficient, so less instructions. Try the

same experiment

without the "-O2", an you will see that the non-optimized version is longer than

the optimized one. The optimizer is smart enough to realize that a right-shift

is better here, so it changes the left shift to a right-shift.

The original author of the code in tclUtf.c was smart enough

to realize this, and I realize it too. We don't want less efficient code in

debug mode (in which the optimizer is disabled).

 

I object to making such a change in existing code, without

realizing what was behind it. You could have asked before.

 

Sorry,

       Jan Nijtmans

Re: [TCLCORE] CFV: TIP 726

[Jan N. <jan...@gm...>]

Op za 30 aug 2025 om 06:43 schreef Ashok:
> Regarding (1) -

I'll come back later on this one.

> Regarding (2) -
>
> It is not clear what you mean by more efficient when the compiled instruction stream is identical at least for clang x64 and gcc arm64. See https://godbolt.org/z/8afa54eh3.  I wrote in the style I did because (in my mind of course) it was more natural and clearer to look up the bit in the mask than shifting the mask to look at the bit. I verified with only two compilers and architectures but see no reason to claim either shift would be more efficient. Compilers are smarter than me.

More efficient means more efficient, so less instructions. Try the
same experiment
without the "-O2", an you will see that the non-optimized version is longer than
the optimized one. The optimizer is smart enough to realize that a right-shift
is better here, so it changes the left shift to a right-shift.
The original author of the code in tclUtf.c was smart enough
to realize this, and I realize it too. We don't want less efficient code in
debug mode (in which the optimizer is disabled).

I object to making such a change in existing code, without
realizing what was behind it. You could have asked before.

Sorry,
       Jan Nijtmans