Re: [TCLCORE] Unicode in Tcl 9 - a commentary and critique

[Jan N. <jan...@gm...>]

Op wo 1 feb. 2023 om 12:14 schreef apnmbx-public--- via Tcl-Core <
tcl...@li...>:

> I’ve updated the document at Unicode in Tcl 9 (magicsplat.com)
> <https://www.magicsplat.com/tcl9/tcl9unicode.html> and included the
> fossil commit id for reference.
>

Thanks!


> I agree that whether -strict is the default or not is a secondary question.
>

OK. Let's set that aside for now.

If you think standard conformant (2) and (3) are not useful, now or in the
> future, then that becomes the point of debate. The argument is whether (1)
> and (5) suffice for all time to come.
>

I'm not saying that (2) and (3) are not useful. They are not on my radar
(never were),
but I won't stand in the way if someone wants to implement it.


> In the current -strict/-nocomplain model, one would likely have to add
> -replace (2), -lossless (3), -discard (6) etc
>
> and the equivalent -encodinglossless 0/1, -encodingreplace 0/1 etc. to
> fconfigure. Obviously, mutually exclusive. This is confusing and not good
> design to have multiple mutually exclusive options.
>

Agreed that it's not a good idea to add more and more flags. Personally, I
don't
like -nocomplain, but I see the need for it. I would be in favor of
deprecating
it (in Tcl 9.1) and removing it (in Tcl 10.0), but now it's no time for
dreaming ;-)

Following TIP 654, the model if not the specifics, we would instead have
>
>
>
> -profile strict, fconfigure -encodingprofile strict for (1)
>
> -profile replace, fconfigure -encodingprofile replace (2)
>
> -profile lossless, fconfigure -encodingprofile lossless (3)
>
> -profile \UXXXX, fconfigure -encodingprofile \UXXXX (4) (meh, not sure I
> like that)
>
> -profile tcl8, fconfigure -encodingprofile tcl8 (5)
>
> -profile discard, fconfigure -encodingprofile discard (6)
>
>
>
> etc.
>
>
>
> which I think is a much cleaner, more extensible interface.
>

I could imagine TIP #654 (or something like this) being implemented
in the future. Adding more flags is not a good idea, I agree, something
like TIP #654 is much cleaner. The -strict/-nocomplain model is not
meant to be complete, to be used until the end of time. It's
simple and serves what most people (least-surprise rule)
expect (I hope).

There is one problem with accepting TIP #654, and that's the
availability of an implementation. Can it be added later? Yes,
I think so, just adding a new "-profile" option and making "-strict"
synonym with "-profile strict". That sounds simple. In practice,
it isn't. The Tcl channel system is quite complicated, showing
its age. Some day it should be replaced, e.g. by ICU or so.
The profiles have to be implemented by _all_ encodings,
that's a hell of a job. I don't see it happening any time soon.

If someone would start implementing TIP #654, I'm there
offering my help. But I'm afraid it would take too much
time. People want Tcl 8.7/9.0 tomorrow, they already
waited too much time. That's the challenge we
are facing.

Hope this helps,
       Jan Nijtmans