tcl-core — Tcl Core Team discussions about Tcl development

Re: [TCLCORE] CFV: TIP #716

[Poor Y. <org...@po...>]

On 2025-05-27 16:35, apnmbx-public--- via Tcl-Core wrote:
> Nathan,
> 
> You're coming fashionably late to a party that has been going on for 
> months and guests are already leaving 😊 but anyways ...

I like shouting at clouds too.

> 
> Sorry, but I have no idea what "fallout" you are talking about.
> 
> [encoding system] returns the default encoding used for channels, exec 
> etc.
> 
> [encoding user] returns the encoding as configured by the user in his 
> platform-specific manner (registry on Windows). This allows correct 
> setting of channels that communicate with programs that use the user's 
> settings (e.g. findstr, icacls etc command line programs that come with 
> Windows).
> 
> The two may or may not be the same.
> 
> The C API parallels the above.
> 
> What is the fallout that users will have to suffer forever?

The addition of [encoding user] leaks platform-specific details (Windows 
in
this case) into a layer that is supposed to abstract such details away.
*nix programmers aren't going to be familiar with the situation, and are 
going
to have to figure out why there is both an [encoding system] and an 
[encoding
user], only to come to the conclusion that on their own platform there's 
no
sensible difference, and [encoding system] now obsolete, but then 
[encoding
user] may or may not be obsolete since it was just added, and who really 
knows
anymore, and even at the C level there's now 
Tcl_GetEncodingNameForUser() and
so we should probably just use Tcl_GetEncodingNameForUser() now and 
never use
Tcl_GetEncodingNameFromEnvironment(), but then why didn't the core team 
just
fix Tcl_GetEncodingNameFromEnvironment() instead of adding a new 
function while
making the old function completely useless?

What we're seeing here is the suboptimal results of design by committee. 
  I
think the vote on TIP 816 should be called off to avoid creating yet 
another
mess.


-- 
Yorick

Re: [TCLCORE] CFV: TIP #716

[Marc C. <cul...@gm...>]

TIP #716: YES

- Marc

On Mon, May 26, 2025 at 1:03 PM apnmbx-public--- via Tcl-Core <
tcl...@li...> wrote:

> This is a Call For Votes for *TIP 716: New command ”encoding user”,
> remove UTF-8 manifest setting on Windows*
>
>
>
> TIP - https://core.tcl-lang.org/tips/doc/trunk/tip/716.md
>
> Branch – tip-716
>
>
>
> Note the TIP targets 9.0.2 because I think it is important to have it
> released while 9.0 is still relatively young. As the TIP mentions, the
> changes to the stubs table will however only target 9.1 for stubs
> compatibility reasons.
>
>
>
> Special thanks to Jan for his thorough critique and fixes.
>
>
>
> Since there has been plenty of discussion already, voting period is
> limited to one week. Voting ends Monday, June 2, 12PM UTC.
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>

Re: [TCLCORE] CFV: TIP #716

[<apn...@ya...>]

Nathan,

You're coming fashionably late to a party that has been going on for months and guests are already leaving 😊 but anyways ...

Sorry, but I have no idea what "fallout" you are talking about.

[encoding system] returns the default encoding used for channels, exec etc.

[encoding user] returns the encoding as configured by the user in his platform-specific manner (registry on Windows). This allows correct setting of channels that communicate with programs that use the user's settings (e.g. findstr, icacls etc command line programs that come with Windows).

The two may or may not be the same.

The C API parallels the above.

What is the fallout that users will have to suffer forever?


/Ashok

-----Original Message-----
From: Poor Yorick <org...@po...> 
Sent: Tuesday, May 27, 2025 12:10 PM
To: tcl...@li...
Cc: apn...@ya...
Subject: Re: [TCLCORE] CFV: TIP #716

On 2025-05-27 06:03, apnmbx-public--- via Tcl-Core wrote:
> While I was tempted to just remove the manifest and revert to 8.6 
> behavior
> that introduces an incompatibility (as you mention) w.r.t 9.0. And 
> while
> that can *perhaps* be tolerated by the user with a release note, the 
> broader
> opinion in the group while this was being discussed was that utf-8 is 
> the
> future on all platforms. If that is the case, then changing back to 
> utf-8
> for 9.1 or .2 or whatever, would once again force an incompatibility by
> reverting to 9.0 behavior. Making the user jump through hoops twice was 
> a
> little much for me. All this was discussed in both the thread and the 
> TIP.

What that would actually look like would be that anyone affected by the
incompatibility (probably only a couple of people since adoption of 9.0 
is
barely underway) would add an unconditional [encoding system utf-8], 
which
would someday be removed when someone perusing the code notices that 
it's no
longer necessary.  But if TIP 716 is adopted then everyone has to deal 
with the
fallout forever.  It's not a good tradeoff.

-- 
Yorick

[TCLCORE] TCL/Tk biweekly telco on 2nd of June, 2025 12:00 UTC on https://meet.jit.si/TclMonthlyMeetup

[Harald O. <har...@el...>]

Dear Tcl/Tk team,

please allow me to invite anybody to the telco:
Where: https://meet.jit.si/TclMonthlyMeetup
When: 2025-06-02 12:00 UTC

Possible agenda:
1) Release items for TCL/Tk 9.0.2
2) Release items for Tcl/Tk 9.1.0a0
3) TIP 723: monotonic clock: is it a bug? Platform solutions
4) TIP 712: positive options to the subst command
5) Reinhards new socket stuff
6) tk print enhancements
7) text widget glyph line break using ICU or not? 
https://core.tcl-lang.org/tk/info/5d0bc3cfec7c1adb
8) TIP 720: TCL compiler reform
9) Documentation
10) Orphan packages repository
11) Conference status

Other meetings:
10th of June 10:00 UTC: monthly meetup
16th of June 12:00 UTC: UTC Monthly Virtual Meetup on same jitsi

Thank you all,
Harald

Re: [TCLCORE] CFV: TIP #716

[Kevin W. <kw...@co...>]

<div><img width="1" height="1" src="https://fedbdhd.r.bh.d.sendibt3.com/tr/op/m3KpZHgGHonYixSnXQ7--awI61gMslaT7DHu803n6913XzHOu2CVpZrkM80E0LXmUOGDSnOZf6gFYKbsLeodniKb8kH5lb89pAwz55xOlf08FnJfVM__MVkKkgcPMhj5UMclg6KO5F__VhjMY4Y6M6V52G_VwP6qQDXwIvoeG84XskS2pRJt2YXkLp_tWpPGSRjmNIdG5BdGv4NfB5q7Hwlu0oJN6-SE" style="mso-hide:all"/></div>TIP 716: YES<br/><br/>On 5/27/25 3:02 AM, Harald Oehlmann wrote:<br/>> Am 26.05.2025 um 20:02 schrieb apnmbx-public--- via Tcl-Core:<br/>>> This is a Call For Votes for /TIP 716: New command ”encoding user”, <br/>>> remove UTF-8 manifest setting on Windows/<br/>> My vote: yes<br/>><br/>> Thanks for the great work,<br/>> Harald<br/>><br/>><br/>> _______________________________________________<br/>> Tcl-Core mailing list<br/>> Tcl...@li...<br/>> https://lists.sourceforge.net/lists/listinfo/tcl-core<br/>

Re: [TCLCORE] tk-print-fixes branch ready for review

[Harald O. <har...@el...>]

Am 26.05.2025 um 17:26 schrieb Emiliano G:
> As some of you might already know, I was working in the tk-print-fixes
> branch trying to fix some issues I've found while converting a custom
> printing solution (involving ghostscript on windows) to the new 9.0
> feature. As I kept finding issues, and reporting them (see the list at
> https://core.tcl-lang.org/tk/ticket?s=tk+print ), soon this workflow
> proved to be cumbersome, so this was the motivation for this branch.
> It has reached a point where I'm satisfied with the current state.
> 
> The changed files are win/tkWinGDI.c and generic/print.tcl , and it
> addresses specifically windows printing, both in [text] and [canvas]
> widget printing. Nothing was modified from the user POV, besides
> output. The only documented API is still [tk print $widget], and the
> rest are implementation details.
> 
> A short, non-comprehensive list of changes, divided in two categories:
> 
> User visible changes:
> * Plain text printing for non-latin characters. This is the main
> visible change for the [text] widget printing, along with the change
> of default font, now being monospaced (Courier New), which makes it
> consistent with *nix printing.
> * Rotated text on canvas now works (with a caveat, see code changes).
> * Correct joinstyle and capstyle for lines.
> * Correct capstyle for arcs (arc style). While not configurable, it
> uses (fixed) butt style.
> * Correct joinstyle for polygon, rectangle and arcs (pieslice and
> chord style); while only polygon accepts a -joinstyle option,
> rectangles use (fixed) miter and arcs use bevel.
> * Correct arrows on lines.
> * Smooth lines support complete, both bezier and raw.
> * Correct management of -outline and -fill color specification. If
> it's specified as {}, it means "don't draw this element".
> * Don't draw items with "hidden" state (consistency with *nix printing).
> 
> Code changes (non visible to users):
> * Fixed memory leaks (ckalloc without ckfree).
> * Protect internal commands from being called with a NULL device context.
> * Rewrite most parsing stages to use Tcl_ParseArgsObjv(). The parsing
> style of item options was both inconsistent (strcmp, strncmp, check of
> correct number of args, etc) and repetitive (same code replicated
> everywhere). IMO this improves maintainability in the long run. Yes,
> I'm being opinionated here :-)
> * Plain text printing (text widget) now uses a different, simpler code
> path than canvas text items. This should be faster for long documents.
> * Moved global state to an interp specific state. Code is now multi-interp safe.
> * Rotated text on canvas uses Tk_ComputeTextLayout() to get the same
> line breaks as the canvas text item with the -width option different
> from zero. I had to copy the relevant struct definitions from
> generic/tkFont.c in order to access its fields. If those definitions
> are moved to generic/tkFont.h , they can be removed and replaced with
> an #include.
> 
> There's still an issue I want to fix: dashes are not implemented
> (yet). Other missing features, like stipples, bitmaps or embedded
> windows are not on my TODO list. Further work involves matching
> expected output of canvas printing between different platforms. Right
> now there's a difference between Windows and *nix (not MacOS) output.
> 
> Please review and test. Thank you all.
> 
> Regards
> 
> Emiliano
Emiliano,
highly appreciated, thank you! I am probably "guilty" for the not 
working unicode text, as I provided the source to Kevin...

Thanks again,
Harald

Re: [TCLCORE] CFV: TIP #716

[Harald O. <har...@el...>]

Am 26.05.2025 um 20:02 schrieb apnmbx-public--- via Tcl-Core:
> This is a Call For Votes for /TIP 716: New command ”encoding user”, 
> remove UTF-8 manifest setting on Windows/
My vote: yes

Thanks for the great work,
Harald

Re: [TCLCORE] CFV: TIP #716

[Jan N. <jan...@gm...>]

Op ma 26 mei 2025 om 20:03 schreef Ashok:
>
> This is a Call For Votes for TIP 716: New command ”encoding user”, remove UTF-8 manifest setting on Windows

TIP #716: Yes

Regards,
       Jan Nijtmans

[TCLCORE] expose content of macOS' Info.plist

[IOhannes m z. <zmo...@ie...>]

(please CC me, as i'm not subscribed to this mailinglist)


i recently asked a question on stackoverflow[1], and donal kindly 
suggested to bring it to the tcl-core ml (if there's a more appropriate 
ml, please do not hesitate to tell me)

TL;DR: it would be great if there was some way to expose the contents of 
the embedded Info.plist of a Tcl/Tk application on the Tcl side. an 
obvious candidate is `::tk::mac`


context: we are building a cross-platform application based on Tcl/Tk 
(that is: Wish). on macOS we ship an app-bundle.
Some people build applications on top of ours, and it would be great if 
they only need to do minimally invasive changes to create an all-new app.

one of the (seemingly trivial) problems we have identified is the 
application *name*, as displayed by the OS, but also within some dialogs.

on macOS, a lot of meta-information about an application is embedded in 
`Content/Info.plist` of the app bundle.
the `CFBundleName` and `CFBundleDisplayName` keys of the info.plist is 
automatically used by macOS for displaying the application's title.

now i would like to access this dictionary from within my Tcl/Tk code, 
so I can use those values in my custom dialogs and menus and whatnot.
(and in order to build a downstream app, I only need to change the 
Info.plist and the rest happens automatically).

we tried reading the Info.plist with macOS's `defaults` tool (which is 
able to read both binary and XML plist files), but this turned out to be 
*absymally* slow (we did this at application startup, and people 
complained verbosely about a 3-5 times longer startup)
afair, the reason for the slow startup is that macOS gatekeeper needs to 
check whether they are actually allowed to access this file - after all, 
`defaults read` just reads some random application's Info.plist which 
might be a security problem or i dunno...)
anyhow, I think that this would not be a problem when accessing the info 
on the ObjectiveC side (`[NSBundle mainBundle]`)

so I would like to suggest to expose the information stored in 
`Info.plist` to my Tcl/Tk scripts.
I don't really care how (dicts, arrays, getter procs)


(sidenote: i can already access the `CFBundleIdentifier` via an env-var 
"__CFBundleIdentifier"; afaict this is something that macOS does on its 
own; it also appears to be the only value from Info.plist that is 
exposed like this)


thanks for considering.

gfmad
IOhannes

PS: please CC me in any replies, as i'm not subscribed to this mailinglist.


[1] https://stackoverflow.com/questions/79628543/

Re: [TCLCORE] TIP 723: monotomic clock for after, independent of TIP 233 (Tcl_SetTimeProc/Tcl_QueryTimeProc), MS-Windows only

[<apn...@ya...>]

Regardless of what we decide in terms of a new command or a “fix” to after, we should absolutely be consistent across platforms in what exactly we are measuring. In my opinion, that includes whether time in suspension is counted or not.

 

Looking the branch tkt3328635-posix-monotonic-clock (I don’t know if this is the right patch to examine), it seems it uses CLOCK_MONOTONIC if available and CLOCK_REALTIME if not.

 

Leaving aside the difference between the two, and the brokenness Marc reports, CLOCK_MONOTONIC itself does not seem consistent across platforms. Linux manpages say it does not include time that the system sleeps. MacOS manpages say it does. So there is that inconsistency within CLOCK_MONOTONIC itself between platforms.

 

Since GetTickCount64 (or whatever it’s called) includes nap time, on Linux perhaps we should be using either CLOCK_BOOTTIME or CLOCK_TAI if available.

 

Sorry, not offering a solution but a plea for consistency.

 

/Ashok 

 

From: Marc Culler <cul...@gm...> 
Sent: Tuesday, May 27, 2025 6:18 AM
To: Harald Oehlmann <har...@el...>
Cc: Tcl Core List <tcl...@li...>
Subject: Re: [TCLCORE] TIP 723: monotomic clock for after, independent of TIP 233 (Tcl_SetTimeProc/Tcl_QueryTimeProc), MS-Windows only

 

According to this page <https://discussions.apple.com/thread/253778121>  Apple supports CLOCK_MONOTONIC but that clock is broken; it is not in fact monotonic.  On the other hand the same page claims that there is a different clock id, CLOCK_MONOTONIC_RAW, which *is* monotonic and which also does not stop when the system is asleep.  So, if this information is correct, to fix the problem on macOS we would only need to switch to using CLOCK_MONOTONIC_RAW.

 

It would be nice to have a test script that could be used on all platforms.

 

- Marc

 

On Mon, May 26, 2025 at 11:00 AM Harald Oehlmann <har...@el... <mailto:har...@el...> > wrote:

Hi Sergey,
thank you for the "very simple explanation". So, there is a system, 
where the wall clock has a 20ms resolution and there is a thread to 
generate a 1ms solution from it by a sort of interpolation, which may 
lead to not hitting the 20ms point and thus interpolates even more - Great!
I am so happy, that the monotonic ms clock is just one call and 
everything is fine...

Is there anything wrong with the TIP?

Thanks for all,
Harald

Am 26.05.2025 um 17:35 schrieb Dipl. Ing. Sergey G. Brester:
> It is very simple:
> 
>   * calibration thread <https://core.tcl-lang.org/tcl/file?
>     ci=f6c559b364487a9d&name=win/tclWinTime.c&ln=743-762> - is an extra
>     thread in tcl (windows only) for time calibration purposes, to keep
>     the high resolution time of perf-counter synchronized with system
>     time (which has lower resolution).
>   * short jumps or drifts - are the drifts occurred by the
>     synchronization if adjustment happens:
>     e. g. if system time will be adjusted, the virtual time (VT)
>     calculated from perf-counter must be adjusted too, but it happens
>     not immediately "correct", because the calibration factor shall be
>     also adjusted slowly due to lower resolution of system time, so VT
>     may near to system clock in steps of max 20ms (minimal resolution of
>     system clock) - so causes the short jumps up to plus/minus 20ms back
>     and forth.
>   * the above was the reason why I removed the calibration thread
>     completely in my RFE (replaced this with better algorithm used perf-
>     counter and mono-time, and calibrated the time within Tcl_GetTime
>     and co periodically and more sane).
> 
> Related the manual adjustment, it'd only explain if the machines would 
> work in UTC/GMT timezone (and adjustment of system clock would affect 
> the virtual UTC-time of tcl too). If the machines would be in the CET 
> (Berlin) time zone, the manual adjustment should not affect UTC-time at 
> all, or at least not in such large values.
> Moreover you'd probably not need the time adjustment at all, because 
> the continuous UTC-time will automatically trigger a DST-switch related 
> to TZdata (e. g. in the same way I provided it above with clock format 
> 1743296399 and next second).
> 
> 26.05.2025 15:25, Harald Oehlmann wrote:
> 
>> Sergey,
>> thanks for all the great explanations. Sounds like high Wizard stuff - time calibration - short jumps - never heard about all this.
>> Please correct the TIP if there is something wrong.
>> I tried after your last comment...

_______________________________________________
Tcl-Core mailing list
Tcl...@li... <mailto:Tcl...@li...> 
https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] CFV: TIP #716

[<apn...@ya...>]

TIP #716: YES

 

From: apnmbx-public--- via Tcl-Core <tcl...@li...> 
Sent: Monday, May 26, 2025 11:33 PM
To: tcl...@li...
Subject: [TCLCORE] CFV: TIP #716

 

This is a Call For Votes for TIP 716: New command "encoding user", remove
UTF-8 manifest setting on Windows

 

TIP - https://core.tcl-lang.org/tips/doc/trunk/tip/716.md

Branch - tip-716

 

Note the TIP targets 9.0.2 because I think it is important to have it
released while 9.0 is still relatively young. As the TIP mentions, the
changes to the stubs table will however only target 9.1 for stubs
compatibility reasons.

 

Special thanks to Jan for his thorough critique and fixes.

 

Since there has been plenty of discussion already, voting period is limited
to one week. Voting ends Monday, June 2, 12PM UTC.

Re: [TCLCORE] CFV: TIP #716

[<apn...@ya...>]

While I was tempted to just remove the manifest and revert to 8.6 behavior
that introduces an incompatibility (as you mention) w.r.t 9.0. And while
that can *perhaps* be tolerated by the user with a release note, the broader
opinion in the group while this was being discussed was that utf-8 is the
future on all platforms. If that is the case, then changing back to utf-8
for 9.1 or .2 or whatever, would once again force an incompatibility by
reverting to 9.0 behavior. Making the user jump through hoops twice was a
little much for me. All this was discussed in both the thread and the TIP.

-----Original Message-----
From: Poor Yorick <org...@po...> 
Sent: Tuesday, May 27, 2025 4:04 AM
To: tcl...@li...
Subject: Re: [TCLCORE] CFV: TIP #716

On 2025-05-27 01:08, Poor Yorick wrote:
[SNIP]
> 
> TIP 816 however goes further and introduces a new function,
> Tcl_GetEncodingNameForUser().  This adds cognitive load, and I don't 
> think it's
> worth it.  If implemented to take the relevant values from the
> environment/registory/platform into account,
> Tcl_GetEncodingNameFromEnvironment() is sufficient,  and in that case 
> [encoding
> system] is also sufficient, even for the [exec -encoding] case.

chatGPT agrees that tweaking Tcl_GetEncodingNameFromEnvironment() to 
handle the
issue is a cleaner fix.  Here's is our conversation:

     https://chatgpt.com/share/6834e9e4-6d20-800d-936f-1c319434d6fc

-- 
Yorick



_______________________________________________
Tcl-Core mailing list
Tcl...@li...
https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] TIP 723: monotomic clock for after, independent of TIP 233 (Tcl_SetTimeProc/Tcl_QueryTimeProc), MS-Windows only

[Marc C. <cul...@gm...>]

According to this page <https://discussions.apple.com/thread/253778121>
Apple supports CLOCK_MONOTONIC but that clock is broken; it is not in fact
monotonic.  On the other hand the same page claims that there is a
different clock id, CLOCK_MONOTONIC_RAW, which *is* monotonic and which
also does not stop when the system is asleep.  So, if this information is
correct, to fix the problem on macOS we would only need to switch to using
CLOCK_MONOTONIC_RAW.

It would be nice to have a test script that could be used on all platforms.

- Marc

On Mon, May 26, 2025 at 11:00 AM Harald Oehlmann <
har...@el...> wrote:

> Hi Sergey,
> thank you for the "very simple explanation". So, there is a system,
> where the wall clock has a 20ms resolution and there is a thread to
> generate a 1ms solution from it by a sort of interpolation, which may
> lead to not hitting the 20ms point and thus interpolates even more - Great!
> I am so happy, that the monotonic ms clock is just one call and
> everything is fine...
>
> Is there anything wrong with the TIP?
>
> Thanks for all,
> Harald
>
> Am 26.05.2025 um 17:35 schrieb Dipl. Ing. Sergey G. Brester:
> > It is very simple:
> >
> >   * calibration thread <https://core.tcl-lang.org/tcl/file?
> >     ci=f6c559b364487a9d&name=win/tclWinTime.c&ln=743-762> - is an extra
> >     thread in tcl (windows only) for time calibration purposes, to keep
> >     the high resolution time of perf-counter synchronized with system
> >     time (which has lower resolution).
> >   * short jumps or drifts - are the drifts occurred by the
> >     synchronization if adjustment happens:
> >     e. g. if system time will be adjusted, the virtual time (VT)
> >     calculated from perf-counter must be adjusted too, but it happens
> >     not immediately "correct", because the calibration factor shall be
> >     also adjusted slowly due to lower resolution of system time, so VT
> >     may near to system clock in steps of max 20ms (minimal resolution of
> >     system clock) - so causes the short jumps up to plus/minus 20ms back
> >     and forth.
> >   * the above was the reason why I removed the calibration thread
> >     completely in my RFE (replaced this with better algorithm used perf-
> >     counter and mono-time, and calibrated the time within Tcl_GetTime
> >     and co periodically and more sane).
> >
> > Related the manual adjustment, it'd only explain if the machines would
> > work in UTC/GMT timezone (and adjustment of system clock would affect
> > the virtual UTC-time of tcl too). If the machines would be in the CET
> > (Berlin) time zone, the manual adjustment should not affect UTC-time at
> > all, or at least not in such large values.
> > Moreover you'd probably not need the time adjustment at all, because
> > the continuous UTC-time will automatically trigger a DST-switch related
> > to TZdata (e. g. in the same way I provided it above with clock format
> > 1743296399 and next second).
> >
> > 26.05.2025 15:25, Harald Oehlmann wrote:
> >
> >> Sergey,
> >> thanks for all the great explanations. Sounds like high Wizard stuff -
> time calibration - short jumps - never heard about all this.
> >> Please correct the TIP if there is something wrong.
> >> I tried after your last comment...
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>

Re: [TCLCORE] CFV: TIP #716

[Poor Y. <org...@po...>]

On 2025-05-27 01:08, Poor Yorick wrote:
[SNIP]
> 
> TIP 816 however goes further and introduces a new function,
> Tcl_GetEncodingNameForUser().  This adds cognitive load, and I don't 
> think it's
> worth it.  If implemented to take the relevant values from the
> environment/registory/platform into account,
> Tcl_GetEncodingNameFromEnvironment() is sufficient,  and in that case 
> [encoding
> system] is also sufficient, even for the [exec -encoding] case.

chatGPT agrees that tweaking Tcl_GetEncodingNameFromEnvironment() to 
handle the
issue is a cleaner fix.  Here's is our conversation:

     https://chatgpt.com/share/6834e9e4-6d20-800d-936f-1c319434d6fc

-- 
Yorick

Re: [TCLCORE] CFV: TIP #716

[Poor Y. <org...@po...>]

On 2025-05-26 21:02, apnmbx-public--- via Tcl-Core wrote:
> This is a Call For Votes for TIP 716: New command "encoding user", 
> remove
> UTF-8 manifest setting on Windows

During the discussions of TIP 816 it became clear that it was a mistake 
to
change the manifest such that the process encoding is hard-coded to 
utf-8.  The
purpose of the change was never clarified, and it seems that the change 
did not
accomplish anything useful.  On the other hand, the change makes Tcl
incompatible with shared libraries that can't handle utf-8.  This change 
was a
mistake and should be reverted.

TIP 816 however goes further and introduces a new function,
Tcl_GetEncodingNameForUser().  This adds cognitive load, and I don't 
think it's
worth it.  If implemented to take the relevant values from the
environment/registory/platform into account,
Tcl_GetEncodingNameFromEnvironment() is sufficient,  and in that case 
[encoding
system] is also sufficient, even for the [exec -encoding] case.

I think that TIP 816 should be simplified to merely revert the change to 
the
manifest, with other improvements like [exec -encoding] moved to a 
separate
TIP.  As has been discussed, just reverting the change to the manifest 
would
mean an incompatibility between Tcl 9.0 and Tcl 9.1, but it's one that 
is
relatively easily dealt with by giving notice to users that 
compatibility with
Tcl 9.0 can be restored via [encoding system utf-8].

-- 
Yorick

[TCLCORE] CFV: TIP #716

[<apn...@ya...>]

This is a Call For Votes for TIP 716: New command "encoding user", remove
UTF-8 manifest setting on Windows

 

TIP - https://core.tcl-lang.org/tips/doc/trunk/tip/716.md

Branch - tip-716

 

Note the TIP targets 9.0.2 because I think it is important to have it
released while 9.0 is still relatively young. As the TIP mentions, the
changes to the stubs table will however only target 9.1 for stubs
compatibility reasons.

 

Special thanks to Jan for his thorough critique and fixes.

 

Since there has been plenty of discussion already, voting period is limited
to one week. Voting ends Monday, June 2, 12PM UTC.

Re: [TCLCORE] TIP 723: monotomic clock for after, independent of TIP 233 (Tcl_SetTimeProc/Tcl_QueryTimeProc), MS-Windows only

[Harald O. <har...@el...>]

Hi Sergey,
thank you for the "very simple explanation". So, there is a system, 
where the wall clock has a 20ms resolution and there is a thread to 
generate a 1ms solution from it by a sort of interpolation, which may 
lead to not hitting the 20ms point and thus interpolates even more - Great!
I am so happy, that the monotonic ms clock is just one call and 
everything is fine...

Is there anything wrong with the TIP?

Thanks for all,
Harald

Am 26.05.2025 um 17:35 schrieb Dipl. Ing. Sergey G. Brester:
> It is very simple:
> 
>   * calibration thread <https://core.tcl-lang.org/tcl/file?
>     ci=f6c559b364487a9d&name=win/tclWinTime.c&ln=743-762> - is an extra
>     thread in tcl (windows only) for time calibration purposes, to keep
>     the high resolution time of perf-counter synchronized with system
>     time (which has lower resolution).
>   * short jumps or drifts - are the drifts occurred by the
>     synchronization if adjustment happens:
>     e. g. if system time will be adjusted, the virtual time (VT)
>     calculated from perf-counter must be adjusted too, but it happens
>     not immediately "correct", because the calibration factor shall be
>     also adjusted slowly due to lower resolution of system time, so VT
>     may near to system clock in steps of max 20ms (minimal resolution of
>     system clock) - so causes the short jumps up to plus/minus 20ms back
>     and forth.
>   * the above was the reason why I removed the calibration thread
>     completely in my RFE (replaced this with better algorithm used perf-
>     counter and mono-time, and calibrated the time within Tcl_GetTime
>     and co periodically and more sane).
> 
> Related the manual adjustment, it'd only explain if the machines would 
> work in UTC/GMT timezone (and adjustment of system clock would affect 
> the virtual UTC-time of tcl too). If the machines would be in the CET 
> (Berlin) time zone, the manual adjustment should not affect UTC-time at 
> all, or at least not in such large values.
> Moreover you'd probably not need the time adjustment at all, because 
> the continuous UTC-time will automatically trigger a DST-switch related 
> to TZdata (e. g. in the same way I provided it above with clock format 
> 1743296399 and next second).
> 
> 26.05.2025 15:25, Harald Oehlmann wrote:
> 
>> Sergey,
>> thanks for all the great explanations. Sounds like high Wizard stuff - time calibration - short jumps - never heard about all this.
>> Please correct the TIP if there is something wrong.
>> I tried after your last comment...

Re: [TCLCORE] TIP 723: monotomic clock for after, independent of TIP 233 (Tcl_SetTimeProc/Tcl_QueryTimeProc), MS-Windows only

[Dipl. I. S. G. B. <se...@us...>]

 

It is very simple: 

 	* calibration thread [1] - is an extra thread in tcl (windows only)
for time calibration purposes, to keep the high resolution time of
perf-counter synchronized with system time (which has lower resolution).

	* short jumps or drifts - are the drifts occurred by the
synchronization if adjustment happens: 
e. g. if system time will be adjusted, the virtual time (VT) calculated
from perf-counter must be adjusted too, but it happens not immediately
"correct", because the calibration factor shall be also adjusted slowly
due to lower resolution of system time, so VT may near to system clock
in steps of max 20ms (minimal resolution of system clock) - so causes
the short jumps up to plus/minus 20ms back and forth. 	* the above was
the reason why I removed the calibration thread completely in my RFE
(replaced this with better algorithm used perf-counter and mono-time,
and calibrated the time within Tcl_GetTime and co periodically and more
sane).

Related the manual adjustment, it'd only explain if the machines would
work in UTC/GMT timezone (and adjustment of system clock would affect
the virtual UTC-time of tcl too). If the machines would be in the CET
(Berlin) time zone, the manual adjustment should not affect UTC-time at
all, or at least not in such large values.
Moreover you'd probably not need the time adjustment at all, because the
continuous UTC-time will automatically trigger a DST-switch related to
TZdata (e. g. in the same way I provided it above with clock format
1743296399 and next second). 

26.05.2025 15:25, Harald Oehlmann wrote: 

> Sergey,
> thanks for all the great explanations. Sounds like high Wizard stuff - time calibration - short jumps - never heard about all this.
> Please correct the TIP if there is something wrong.
> I tried after your last comment...
 

Links:
------
[1]
https://core.tcl-lang.org/tcl/file?ci=f6c559b364487a9d&name=win/tclWinTime.c&ln=743-762

[TCLCORE] tk-print-fixes branch ready for review

[Emiliano G <emi...@gm...>]

As some of you might already know, I was working in the tk-print-fixes
branch trying to fix some issues I've found while converting a custom
printing solution (involving ghostscript on windows) to the new 9.0
feature. As I kept finding issues, and reporting them (see the list at
https://core.tcl-lang.org/tk/ticket?s=tk+print ), soon this workflow
proved to be cumbersome, so this was the motivation for this branch.
It has reached a point where I'm satisfied with the current state.

The changed files are win/tkWinGDI.c and generic/print.tcl , and it
addresses specifically windows printing, both in [text] and [canvas]
widget printing. Nothing was modified from the user POV, besides
output. The only documented API is still [tk print $widget], and the
rest are implementation details.

A short, non-comprehensive list of changes, divided in two categories:

User visible changes:
* Plain text printing for non-latin characters. This is the main
visible change for the [text] widget printing, along with the change
of default font, now being monospaced (Courier New), which makes it
consistent with *nix printing.
* Rotated text on canvas now works (with a caveat, see code changes).
* Correct joinstyle and capstyle for lines.
* Correct capstyle for arcs (arc style). While not configurable, it
uses (fixed) butt style.
* Correct joinstyle for polygon, rectangle and arcs (pieslice and
chord style); while only polygon accepts a -joinstyle option,
rectangles use (fixed) miter and arcs use bevel.
* Correct arrows on lines.
* Smooth lines support complete, both bezier and raw.
* Correct management of -outline and -fill color specification. If
it's specified as {}, it means "don't draw this element".
* Don't draw items with "hidden" state (consistency with *nix printing).

Code changes (non visible to users):
* Fixed memory leaks (ckalloc without ckfree).
* Protect internal commands from being called with a NULL device context.
* Rewrite most parsing stages to use Tcl_ParseArgsObjv(). The parsing
style of item options was both inconsistent (strcmp, strncmp, check of
correct number of args, etc) and repetitive (same code replicated
everywhere). IMO this improves maintainability in the long run. Yes,
I'm being opinionated here :-)
* Plain text printing (text widget) now uses a different, simpler code
path than canvas text items. This should be faster for long documents.
* Moved global state to an interp specific state. Code is now multi-interp safe.
* Rotated text on canvas uses Tk_ComputeTextLayout() to get the same
line breaks as the canvas text item with the -width option different
from zero. I had to copy the relevant struct definitions from
generic/tkFont.c in order to access its fields. If those definitions
are moved to generic/tkFont.h , they can be removed and replaced with
an #include.

There's still an issue I want to fix: dashes are not implemented
(yet). Other missing features, like stipples, bitmaps or embedded
windows are not on my TODO list. Further work involves matching
expected output of canvas printing between different platforms. Right
now there's a difference between Windows and *nix (not MacOS) output.

Please review and test. Thank you all.

Regards

Emiliano

Re: [TCLCORE] TIP 723: monotomic clock for after, independent of TIP 233 (Tcl_SetTimeProc/Tcl_QueryTimeProc), MS-Windows only

[Harald O. <har...@el...>]

Sergey,
thanks for all the great explanations. Sounds like high Wizard stuff - 
time calibration - short jumps - never heard about all this.
Please correct the TIP if there is something wrong.
I tried after your last comment...

My application is very simple. The machines are in pharamceutical 
production, network free.
The setup procedure to a new production step always requires a manual 
clock set. I don't know in which time zone or location they are set-up.
Perhaps, they don't match the current time-zone. This is not important 
and not, what is asked. The systems should not automatically adjust DST. 
This is done manually.
Pharmaceutical production is sometimes strange...
Thanks again,
Harald

Am 26.05.2025 um 14:23 schrieb Dipl. Ing. Sergey G. Brester:
> I don't understand the issue with manual adjustment of clock.
> Even if it is done fully manually (without the NTP), the clock (UTC) 
> shall be consistent, because otherwise you'd immediately observe 1 hour 
> difference even in TZ CET (see my examples with 1743296399). So the 
> adjustment of system time ±1 hour shall happen in current TZ (CET/CEST), 
> what shall reflect to few seconds maximally in the UTC time (what 
> doesn't jumps normally). The same is basically valid for clock 
> milliseconds or clock microseconds.
> Just I didn't inspect yet how windows part of tcl would consider it, 
> because it happens in calibration thread there (only windows thing, *nix 
> doesn't have it), which I completely removed in my RFE, due to similar 
> issues (but short jumps), since the calibration was affected by timing 
> issue and by jumps it would near the real time of system slow and can 
> deviate few seconds long unless gets fully synchronous. This was the 
> main reason why I removed the calibration thread in my RFE and rewrote 
> the calibration with different algorithm, that can drift more accurately 
> (only in single direction).
> Maybe this is the issue you had? But it shall be definitely not ±1 hour 
> jumps (in UTC time), but small jumps.
> 
> Hope this helps,
> Sergey.
> 
> 23.05.2025 08:27, Harald Oehlmann wrote:
> 
>> ...
>> About the "summer time change". The concerned systems run without NTP and the change is done manually. So, the clock is manually turned back by one hour once per year.
>> This may be detailed in the tip.
>>
>> Thanks for all,
>> Harald
>>
>>
>> Am 22.05.2025 um 17:38 schrieb Dipl. Ing. Sergey G. Brester:
>>> I don't know why it shall be limited to Windows only, because all 3 
>>> points are affecting *nix in the same way:   * manual time adjustment 
>>> (is possible with `date -s`)   * suspension of the system (suspend of 
>>> VM, hibernation, etc)   * automatic time adjustment (via ntpd and co) 
>>> Also there is a mistake in the TIP - "specially on day time switching 
>>> +/- 1 hour". Because DST-switches normally doesn't adjust the clock 
>>> in UTC (GMT), which is running continuously. Everything what asking 
>>> [clock seconds], [clock milliseconds], [clock microseconds] (or 
>>> related C-API functions) etc is not affected by DST-switches at all. 
>>> The DST-switch happens only in related timezone (table of offsets 
>>> stored in tzdata) and doesn't exist without TZ context (pure UTC 
>>> time). Just for illustration: # forward DST jump in CET-TZ: % clock 
>>> format [expr {1743296399 + 0}] -timezone :Europe/Berlin Sun Mar 30 
>>> 01:59:59 CET 2025 % clock format [expr {1743296399 + 1}] - 
>>> timezone :Europe/Berlin Sun Mar 30 03:00:00 CEST 2025 # backward DST 
>>> jump in CET-TZ: % clock format [expr {1761440399 + 0}] - 
>>> timezone :Europe/Berlin Sun Oct 26 02:59:59 CEST 2025 % clock format 
>>> [expr {1761440399 + 1}] -timezone :Europe/Berlin Sun Oct 26 02:00:00 
>>> CET 2025 The issues are only manual or automatic time adjustments 
>>> affecting UTC- time, and suspension/hibernation. However as already 
>>> said it definitely affecting all systems that using wall clock (and 
>>> need a switch to monotonic time). Regards, Sergey. 21.05.2025 12:35, 
>>> Harald Oehlmann wrote:
>>>> Dear TCL team, please allow me to propose a TIP to cure one of my 
>>>> biggest issues on TCL and MS-Windows: after does not fire if the 
>>>> wall clock changes. The TIP text is here: https://core.tcl-lang.org/ 
>>>> tips/doc/trunk/tip/723.md <https://core.tcl-lang.org/tips/doc/trunk/ 
>>>> tip/723.md> <https://core.tcl-lang.org/tips/doc/trunk/tip/723.md 
>>>> <https://core.tcl-lang.org/tips/doc/trunk/tip/723.md>> I am sorry to 
>>>> write it, as it is written (e.g. no respect on TIP233, Windows 
>>>> only). But those are the conclusions taken on recent discussions on 
>>>> the forelast biweekly telco and on the related bug tickets and on 
>>>> the core list. I hope we will find a better solution, but this is my 
>>>> "minimum viable outcome" to this very urgent and annoying issue. 
>>>> Thanks for all and take care, Harald

Re: [TCLCORE] TIP 723: monotomic clock for after, independent of TIP 233 (Tcl_SetTimeProc/Tcl_QueryTimeProc), MS-Windows only

[Dipl. I. S. G. B. <se...@us...>]

 

I don't understand the issue with manual adjustment of clock. 
Even if it is done fully manually (without the NTP), the clock (UTC)
shall be consistent, because otherwise you'd immediately observe 1 hour
difference even in TZ CET (see my examples with 1743296399). So the
adjustment of system time ±1 hour shall happen in current TZ (CET/CEST),
what shall reflect to few seconds maximally in the UTC time (what
doesn't jumps normally). The same is basically valid for clock
milliseconds or clock microseconds.
Just I didn't inspect yet how windows part of tcl would consider it,
because it happens in calibration thread there (only windows thing, *nix
doesn't have it), which I completely removed in my RFE, due to similar
issues (but short jumps), since the calibration was affected by timing
issue and by jumps it would near the real time of system slow and can
deviate few seconds long unless gets fully synchronous. This was the
main reason why I removed the calibration thread in my RFE and rewrote
the calibration with different algorithm, that can drift more accurately
(only in single direction).
Maybe this is the issue you had? But it shall be definitely not ±1 hour
jumps (in UTC time), but small jumps. 

Hope this helps,
Sergey. 

23.05.2025 08:27, Harald Oehlmann wrote: 

> ...
> 
> About the "summer time change". The concerned systems run without NTP and the change is done manually. So, the clock is manually turned back by one hour once per year.
> This may be detailed in the tip.
> 
> Thanks for all,
> Harald
> 
> Am 22.05.2025 um 17:38 schrieb Dipl. Ing. Sergey G. Brester:
> I don't know why it shall be limited to Windows only, because all 3 points are affecting *nix in the same way: * manual time adjustment (is possible with `date -s`) * suspension of the system (suspend of VM, hibernation, etc) * automatic time adjustment (via ntpd and co) Also there is a mistake in the TIP - "specially on day time switching +/- 1 hour". Because DST-switches normally doesn't adjust the clock in UTC (GMT), which is running continuously. Everything what asking [clock seconds], [clock milliseconds], [clock microseconds] (or related C-API functions) etc is not affected by DST-switches at all. The DST-switch happens only in related timezone (table of offsets stored in tzdata) and doesn't exist without TZ context (pure UTC time). Just for illustration: # forward DST jump in CET-TZ: % clock format [expr {1743296399 + 0}] -timezone :Europe/Berlin Sun Mar 30 01:59:59 CET 2025 % clock format [expr {1743296399 + 1}] -timezone :Europe/Berlin Sun Mar 30 03:00:00 CEST 2025 #
backward DST jump in CET-TZ: % clock format [expr {1761440399 + 0}] -timezone :Europe/Berlin Sun Oct 26 02:59:59 CEST 2025 % clock format [expr {1761440399 + 1}] -timezone :Europe/Berlin Sun Oct 26 02:00:00 CET 2025 The issues are only manual or automatic time adjustments affecting UTC- time, and suspension/hibernation. However as already said it definitely affecting all systems that using wall clock (and need a switch to monotonic time). Regards, Sergey. 21.05.2025 12:35, Harald Oehlmann wrote: Dear TCL team, please allow me to propose a TIP to cure one of my biggest issues on TCL and MS-Windows: after does not fire if the wall clock changes. The TIP text is here: https://core.tcl-lang.org/tips/doc/trunk/tip/723.md [1] <https://core.tcl-lang.org/tips/doc/trunk/tip/723.md [1]> I am sorry to write it, as it is written (e.g. no respect on TIP233, Windows only). But those are the conclusions taken on recent discussions on the forelast biweekly telco and on the related bug tickets and on
the core list. I hope we will find a better solution, but this is my "minimum viable outcome" to this very urgent and annoying issue. Thanks for all and take care, Harald
 

Links:
------
[1] https://core.tcl-lang.org/tips/doc/trunk/tip/723.md

Re: [TCLCORE] TIP 723: monotomic clock for after, independent of TIP 233 (Tcl_SetTimeProc/Tcl_QueryTimeProc), MS-Windows only

[Christian W. <Chr...@t-...>]

On 05/25/2025 11:27 AM, Harald Oehlmann wrote:

Harald,

> …
> I can only say, that I do my best of the 3 wished actions:
> - the numeric patch is in
> - the blockcursor was even noticed by Francois, mega-great!
> - the monotonic clock may come on Windows, other platforms: no sponsor, sorry
> - SQLite break and dict patch in

again, I am grateful that you brought these points forward.

Thank you,
Christian

Re: [TCLCORE] TIP 723: monotomic clock for after, independent of TIP 233 (Tcl_SetTimeProc/Tcl_QueryTimeProc), MS-Windows only

[Massimo M. <mas...@ri...>]

On 5/25/25 11:27, Harald Oehlmann wrote:
> Am 24.05.2025 um 21:14 schrieb Christian Werner:
>> On 05/24/2025 07:49 PM, Harald Oehlmann wrote:
>>
> I can only say, that I do my best of the 3 wished actions:
> - the numeric patch is in
> - the blockcursor was even noticed by Francois, mega-great!
> - the monotonic clock may come on Windows, other platforms: no sponsor, 
> sorry

I think having uniform behavior across different platform is important 
on such fundamental feature such as the way Tcl handles time, it's part 
of what defines Tcl, as someone already pointed out

I'm very much under pressure in these weeks and I can't review the patch 
to see what implies for at least the Linux world but we may talk about 
it when we'll hopefully meet in Bologna

  -- Massimo

Re: [TCLCORE] TIP 723: monotomic clock for after, independent of TIP 233 (Tcl_SetTimeProc/Tcl_QueryTimeProc), MS-Windows only

[<apn...@ya...>]

I have not seen any objections, other than from @kbk (iiuc) who was willing to overlook the compatibility concerns with "after" if everyone else was willing to view the change as a bug fix. So presumably we can agree that the clock used for measuring intervals for "after" purposes should be (a) monotonic, and (b) measure elapsed time (so suspended systems keep the clock running).

If not, please speak up now.

So the only concern with regard to "after" is platform consistency. For that, Harald is driving the Windows effort, and the need is for someone to do the same on the other platforms. I understand Sergey and Christian have both provided patches. But I think the issue is that even with patches, there is at least some work involved in reviewing and integrating the patches, tests and verifying CI etc. The hold up is who will volunteer for the task.

Is my understanding of the status correct? If so, I can volunteer for the task though I am really not the right person and hope someone else more knowlegeable will step up.

As far as "interp limit" is concerned, I suspect "cpu cycles" used is the intended use but considering system suspension is a rare case, I can live with it using elapsed time as well.

/Ashok

-----Original Message-----
From: Harald Oehlmann <har...@el...> 
Sent: Sunday, May 25, 2025 5:49 PM
To: tcl...@li...
Subject: Re: [TCLCORE] TIP 723: monotomic clock for after, independent of TIP 233 (Tcl_SetTimeProc/Tcl_QueryTimeProc), MS-Windows only

Am 25.05.2025 um 13:40 schrieb Gustaf Neumann (sslmail):
> 
> 
>> I am only sponsoring "after for monotonic clock on WIndows". This is a critical patch absolutely needed and hits each user each day, probably without noticing the reason.
> 
> maybe i have missed the argument: why does it hit windows users more than unix/macos users?
> -g
I did not say, that it hits Linux users more often than Windows users.
I can not say anything on Unix or Mac-OS.
I can only say, that it hits Windows users.

Any insight on platform Unix/Linux or MacOS welcome. I can not help.

Sorry,
Harald

Re: [TCLCORE] TIP 723: monotomic clock for after, independent of TIP 233 (Tcl_SetTimeProc/Tcl_QueryTimeProc), MS-Windows only

[Harald O. <har...@el...>]

Am 25.05.2025 um 13:40 schrieb Gustaf Neumann (sslmail):
> 
> 
>> I am only sponsoring "after for monotonic clock on WIndows". This is a critical patch absolutely needed and hits each user each day, probably without noticing the reason.
> 
> maybe i have missed the argument: why does it hit windows users more than unix/macos users?
> -g
I did not say, that it hits Linux users more often than Windows users.
I can not say anything on Unix or Mac-OS.
I can only say, that it hits Windows users.

Any insight on platform Unix/Linux or MacOS welcome. I can not help.

Sorry,
Harald