Re: [TCLCORE] Unicode in Tcl 9 - a commentary and critique

[<apn...@ya...>]

Jan,

 

Regarding the version of Tcl tested – I’m sure it was trunk (not any branch) but not sure about exact repository version. I updated my pool post write up so I’m not sure there’s a way to tell and not worth the time to bisect. But I tried it again downloading the following file:

 

https://sourceforge.net/projects/magicsplat/files/barebones-tcl/tcl9.0-dev/tclsh9.0alpha-dev-snapshot-win-x64-sfe-20230117.exe/download

 

which is the Jan 17 build artifact from the Tcl github actions.

 

% encoding convertto -strict ascii \uE0

?

 

So I would assume that this behavior was exhibited at least as of Jan 17. Perhaps it has been fixed in the repository since then. Or your work tree has other changes not committed. I wanted to keep a stable pool while I plowed through my write-up so have not picked up the latest changes, if any.

 

In any case, I think that issue is relatively minor and may be regarded simply as a bug to be fixed. I would prefer to see the larger issues in the document addressed.

 

With regard to not handling Tcl 8.7, let me reiterate the purpose of my write-up. It expresses *my* views on how Tcl should behave wrt Unicode handling accompanied by a rationale and contrast it with how Tcl 9 actually behaves. The intent was to draw more people outside of yourself, Rolf, Harald and Nathan into the discussion. I don’t think filing tickets and bug reports accomplishes that. My belief is there should first be some consensus (or a majority opinion at least) on the desired *behavior* before looking into how it can be implemented in 9 or 8.7. (As an aside, your remark also reflects on the impact 8.7 has on 9.0 in terms of brain cycles consumed as well as a non-optimal end result.)

 

With respect to bug reports and patches as you suggest, as I have stated several times before, I am happy to contribute patches and bug fixes. Neither I, nor others (Nathan already has branches implementing his opinions) expect you to do all the work (you’ve already done far more than can be expected and there would be no Tcl 9 without your effort, so thank you). But first there needs to be agreement on the issues I and others have raised. If you, and through their silence, the TCT have pretty much decided that the current -strict et al design (which for example, is not just a “bug fix”) is a done deal, there is no point in raising tickets or contributing patches. Or further TIP’s for that matter.

 

/Ashok

 

From: Jan Nijtmans <jan...@gm...> 
Sent: Saturday, January 28, 2023 2:52 AM
To: tcl...@li...
Subject: Re: [TCLCORE] Unicode in Tcl 9 - a commentary and critique

 

Op vr 27 jan. 2023 om 16:37 schreef apnmbx-public:

I’ve written up my view of “state of Unicode in Tcl 9” at https://www.magicsplat.com/tcl9/tcl9unicode.html

 

>From this document:

    % encoding convertto -strict ascii \uE0
    ?

    The character U+E0 is not representable in ASCII so in the presence of the -strict option, the above command should have raised an exception

Reading this, I was about to file a Bug report. But then I tried it:

    $ tclsh9.0

    % encoding convertto -strict ascii \uE0

    unexpected character at index 0: 'U+0000E0'

 

 

So, I am wondering which version of Tcl 9.0 you were testing.

 

My other remark is that you don't handle Tcl8.7. One of the

challenges in the TIP's was to provide a smooth upgrade path

from Tcl 8.6 through 8.7 to 9.0. Therefore, -strict sometimes

does nothing in Tcl 9.0, which might be for 2 different

reasons:

   1) The encoding in Tcl 8.6 already did all the 'strict'

       checks, there's nothing to be added specifying -strict

    2) Outputting lone surrogates is simply illegal in

        utf-8/-16/-32,  therefore Tcl 9.0 always throws an

        exception whether -strict is specified or not.

 

I'm sure that this reaction will trigger other reactions ..... that's

fine. But I hope that real inconsistencies will lead to bug-reports,

or (even better) patch submissions. You all are capable to

report more complaints than I can handle. Many are

simply wrong (in my opinion, but you have the right

to disagree on that), but I'm doing my best to filter the

real ones out if it. Reporting bugs multiple times doesn't

help to get them fixed faster, neither duplicate them

on the Tcl Core list and the Tcl chat.

 

Have a nice weekend!

          Jan Nijtmans