Re: [TCLCORE] More on I/O with Tcl 9

[Poor Y. <org...@po...>]

On 2023-01-09 14:31, Donal Fellows wrote:
...
> The main use case for a failure to convert to/from an encoding is when 
> the application has a non-standard way of handling this: in the case of 
> the code that opened the can of worms in the first place, it was the 
> (awful!) convention inside ZIP files where strings can be either UTF-8 
> or ISO 8859-1 (or whatever random vendor encoding was in use on the 
> generating system!) without metadata indicating what was actually used. 
> In that case, the default strategy of 8, replacing impossible 
> characters with ? or U+00FFFD or whatever (as specified by the 
> encoding), is unsuitable. To me, that smacks of being the sort of thing 
> you apply when using the encoding command explicitly, not in I/O 
> channels or working with filenames. But someone opting in for a channel 
> is still fine.
> 

If this is awful, then so are posix filesystems, which operate precisely 
the
same way:  Create some files on a posix filesystem on a portable drive, 
move
the drive to another computer, and the same issues can arise.  This 
issue isn't
limited to zip files or posix filesystems either:  Structured documents 
in
formats such as csv and xml get created in a variety of ways, and even 
where
format specifications require a certain encoding, it's entirely possible 
for the
need to decode from another encoding to arise.  The diversity of 
existing
encodings, even among structured data formats, is why I think "-strict" 
must be
the default in Tcl 9.  This is a change that can be dealt with almost
mechanically when migrating a large code base from Tcl 8 to Tcl 9, so 
the
burden is not as great as some other argument make make it seem to be.
Furthermore, encoding/decoding behaviour in Tcl 8 is so broken that any 
attempt
to remain backwards-compatible with it is crippling to Tcl 9, just as 
the
broken behaviour in Tcl 8 has been crippling to the cause of Tcl.
Encoding/decoding issues are a dealbreaker to any project assessing a
programming language for potential use.

-- 
Yorick