Re: [TCLCORE] CFV warning: TIP #699: Eliminate encoding alias "binary"...

[Poor Y. <org...@po...>]

On 2024-06-26 00:07, Brian Griffin wrote:
[SNIP]
> 
> When dealing with strings, in a language where everything is a string,
> this is perfectly reasonable.
> However, not all data are strings.  When there is a need to process
> and pass non-string data, there needs to be a way to manage this in
> Tcl. The existing (8.6) "binary" implementation more-or-less works
> when needed. Binary data is not strings, so we have to stop discussing
> it within the context of strings.

At the implementation level lists are not strings, dictionaries are not
strings and other values are not strings.  Your argument that binary
data are not strings is true at the implementation level, but it is
false at the script level.  The fundamental abstraction of Tcl is that
everything is a string, and that means that binary data are strings.
You spoke earlier about leaking concepts into the script level that
should remain implementation details.  Many Tcl programmers have the
false impression "binary" is is some sort of shady hack that "allows
binary data to pass through the script level unscathed."  I had that
impression for a few years when I picked up Tcl.  The truth is that it's
not a shady hack, but an elegant design:  At the script level the
difference between "binary" and "string" data melts away because the
string abstraction naturally accommodates binary data.

You argued that removing "binary" as an encoding presents a leak of an
implementation detail to the script level.  That's backwards.  The fake
"binary" encoding is the leak of the implementation detail that
sometimes a string can be represented as an array of single-byte octets.
Even if the same string is represented as an array of 4-byte code
points, it's still a sequence of bytes for all command procedures that
interpret their arguments as sequences of bytes, i.e. as "binary"
values.  The internal type of a "binary" string value can be anything at
all, and command procedures that interpret their data as bytes will
produce exactly the same results from them.  There is an appropriate
place to introduce interpretations of strings, and individuals
procedures like [list] and [dict] are the right place.  A fake encoding
that implies a value type other than string is the wrong place.

At the script level we should absolutely be discussing binary data in
the context of strings, because strings are all we have, and when we
consistently drive that point home throughout the documentation,
programmers can learn to truly understand Tcl.  Understanding Tcl is a
surprisingly long journey given the simplicity of the language.  One
thing that makes the journey longer is bad practice that panders to
misunderstanding in the name of "being helpful".  Tcl has a long track
record of being unhelpfully "helpful", and it's been a long march to
correct the mistakes of the past.  It's worth it to correct them.

-- 
Yorick