Re: [TCLCORE] TIP 716 ready for comments

[Ashok N. <apn...@ya...>]

Jan,

Let me first respond to your second reservation below regarding Tcl_GetEncodingNameForUser().

Since your objection merely stated "it's a bad idea" without elaborating as to why, I have to make some guesses as to what exactly you object to. Based on your TIP 718 which suggests a new function TclWinGetUserEncoding(), I assume you would prefer the function return a Tcl_Encoding as opposed to a character string for the encoding name. I did think about that too, but there are two reasons I went with the current function signature returning the encoding name as a string. The first is that it is consistent with the existing API Tcl_GetEncodingFromEnvironment() which also returns a string, not a Tcl_Encoding. There is something to be said for consistency when two API's have similar functionality. Secondly, while a Tcl_Encoding may be more convenient for Tcl_UtfTo* and friends, it is less so for channel operations. Tcl_SetChannelOption takes encoding names, not Tcl_Encoding as the values for the -encoding option, so you can say

Tcl_SetChannelOption(chan, "-encoding", Tcl_GetEncodingNameForUser(&ds))
(or something similar)

Given Tcl_Encoding and the encoding name can be converted in both directions, it is six of one and half-a-dozen of the other, one being easier for one set of Tcl functions and the other for another set. I went with the encoding name because it is the more fundamental operation and consistent with Tcl_GetEncodingFromEnvironment(). Having said that I have no objection to providing both.

Of course, the above does not apply if your comment about Tcl_GetEncoding... being a bad API was based on some other factor I have not considered. Can't really tell unless you are more specific.

Now, with regard to the mingw msvcrt comment...

It seems to me from your comment about GetACP() in MingW that you think I was suggesting the manifest would not work in MingW. That is not so. GetACP() will work the same way and return utf-8 in the presence of a manifest but the problem is MSVCRT does not support UTF-8.

I do not know if you read the link that I had referenced in the TIP https://www.msys2.org/docs/environments/#msvcrt-vs-ucrt

To quote from there - It doesn't support the UTF-8 locale ("It" being MSVCRT)

When the official release document says MSVCRT does not support UTF-8, I take that at face value. That is why I did not bother to test it. Presuming the tests you reference are the ones you added for TIP 716, they indicate that UTF-8 works for one particular configuration of mingw for one version of gcc for two specific functions (fopen and chmod I think) for one specific west european character. You are free to presume that means msvcrt has no issues with UTF-8. I, however, prefer to go with the official statement and not presume success of one small test case is proof.

So I will keep that reference in the TIP. I can however add your comment that despite the official MSys position you believe MSVCRT works fine based on the tests you did.

As an aside, I do not understand why you are raising the issue of passing FILE* between different C runtimes? When did I ever raise that issue and what does it have to do with the current TIP 716 discussion? Don't we have enough confusion without adding completely irrelevant considerations?

/Ashok




________________________________
From: Jan Nijtmans
Sent: Monday, May 5, 2025 3:20 PM
To: apn...@ya...
Cc: tcl...@li...
Subject: Re: [TCLCORE] TIP 716 ready for comments

Op ma 14 apr 2025 om 06:52 schreef Ashok:
>
> TIP 716: New command "encoding user", remove UTF-8 manifest setting on Windows is ready for comments. It proposes reversion of a change made in 9.0 to tclsh and wish Windows manifests while keeping compatibility with 9.0.{0,1}.

Let me share two more reservations on the TIP text:
1) Two times in the TIP, "MingW msvcrt" builds are mentioned as
    behaving differently from ucrt builds. Quoting:
        "The key difference with respect to the current implementation
issue that this does not impact extensions that call GetACP solving
the first issue listed above, or using MingW msvcrt builds."
    "An example is components built with MingW64 gcc using the msvcrt runtime"

    This is not correct. I did test that (see ticket
           https://core.tcl-lang.org/tcl/tktview/8ffd8cabd1)
     which was done using a MingW msvcrt build. The
     The GetACP() function functions exactly the same.
     There is an issue mixing different runtimes, but that's
     related to using stdin/stdout: different runtimes each
     have their own FILE implementation. So, opening
     a FILE in an extension using UCRT and writing it
     from other extension using MSVCRT is expected
     to crash. My suggestion: remove it from the TIP
     text (or provide a testcase proving your point, I
     will be happy to try it in my environment)

2) There is no usecase for exporting Tcl_GetEncodingNameForUser()
     I tried to use it, but the way it is now it's a bad idea. I am
     thinking about a separate TIP for a more useful approach.
     Stay tuned.

The reason I suspect that you didn't do any tests in the
MinGW msvcrt environment is that compilation in this
environment failed:
    <https://github.com/tcltk/tcl/actions/runs/14725688360>
I corrected that here, as a free service ;-)
     <https://core.tcl-lang.org/tcl/info/479fc6ad0dff8cdd>

Hope this helps,
          Jan Nijtmans