[Tcl-bugs] [ tktoolkit-Bugs-2546087 ] Tk console drops chars after \0

[SourceForge.net <no...@so...>]

Bugs item #2546087, was opened at 2009-01-29 17:41
Message generated for change (Comment added) made by patthoyts
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=112997&aid=2546087&group_id=12997

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: 54. [console]
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Mark Janssen (mpc_janssen)
Assigned to: Chengye Mao (chengyemao)
Summary: Tk console drops chars after \0

Initial Comment:
The Tk console drops characters after a \0 character. Example:

(MJanssen) 3 % set a "abcd\0efgh"
abcd
(MJanssen) 4 % puts $a
abcd
(MJanssen) 5 % string length $a
9
(MJanssen) 6 % 

inserting the same string in a text widget will show all the characters.

Tk version 8.5.4

----------------------------------------------------------------------

>Comment By: Pat Thoyts (patthoyts)
Date: 2009-02-19 00:47

Message:
The offending commit occurred between 8.4.6 and 8.4.7 on Tcl - rmax' commit
of 2004-05-27 (git id is 09528850aca...) changed the encoding handling of
utf-8 encoded channels to stop them emitting invalid utf-8. The internal
representation of \0 in tcl's utf8 is \uc080 (an invalid utf-8 sequence).
The previous implementation leaked this internal representation and a side
effect of this was to end up inserting \uc080 into the stream. This is why
it works now when you add \0 into the text widget I think.
A really quick solution if we want to preserve the consoles original
effect is to add a string map into the console.tcl:tk::ConsoleOutput
function which is called from the Tk C code when stuff is written to the
channel. The channel does pass this procedure the full string including any
embedded nuls (as checked using string length). So adding a [string map {\0
\ufffc} $string] into the text widget insert statement might be the way
forward.


----------------------------------------------------------------------

Comment By: Pat Thoyts (patthoyts)
Date: 2009-02-18 16:08

Message:
This seems to have occurred sometime during 8.4 development. The 8.2.3
console will show a box for the \0 char. 8.4.1-8.4.6 show a space in place
of the \0. In The 8.4.16 truncates at that point. I imagine this will be
something to do with the console channel that is involved on the Windows
console.
A fast workaround is [string map {\0 .} $s]

----------------------------------------------------------------------

Comment By: Mark Janssen (mpc_janssen)
Date: 2009-01-29 17:44

Message:
Forgot to mention this is on windows. (Vista)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=112997&aid=2546087&group_id=12997