iso2022-jp conversion problems

tcl8.4a4 addressed several problems around the
iso2022-jp enconding.
For example, bugs that I submitted in the past was
mostly fixed.
[ BugID: 218099 ] iso2022-jp encoding does not work.
[ BugID: 219283 ] iso2022-jp encoding is broken

However, it still have problems when I convert
relatively long (longer
than several kilo-bytes) japanese texts (eg. Unix
Japanese Manual
Pages) into iso2022-jp. I'll attach a scipt to
reproduce that.

Some details follow.

(1) euc-jp to iso2022-jp gets-puts conversion

When I convert a text with "tclsh8.4 eucjis.tcl -eucjis
-gets infile outfile",
sometimes "esc ( B" is missing, sometimes extra "esc (
B" appears.
While extra "esc ( B" does not matter, missing "esc (
B" causes
missing characters on reading. The error is
reprodusible if I use the
same file, but I don't know how and when it happens.

"od -x -a" of an example error is below. If I extract
the erroneous
line, the error does not occur. Thus the error is not
the code
dependent but context dependent.

[ output from eucjis.tcl -eucjis -gets euc.txt
jis-n3.txt ]

% H $ 7 $ ^ esc ( B nl sp
sp sp sp sp sp
0007760 241b 2442 2139 1b23 4228
0a0a 2020 752d
esc $ B $ 9 ! # esc ( B nl
nl sp sp - u
! 0010000 2020 241b 2542 213d 253c
2148 2d4a 2074
! sp sp esc $ B % = ! < % H
! J - t sp
! 0010020 241b 2442 3b48 4d48 2451
2439 246b 2448
! esc $ B $ H ; H M Q $ 9
$ k $ H $

[ correct output produced from a software called nkf ]

% H $ 7 $ ^ esc ( B nl sp
sp sp sp sp sp
0007760 241b 2442 2139 1b23 4228
0a0a 2020 752d
esc $ B $ 9 ! # esc ( B nl
nl sp sp - u
! 0010000 2020 241b 2542 213d 253c
2148 1b4a 4228
! sp sp esc $ B % = ! < % H
! J esc ( B
! 0010020 742d 1b20 4224 4824 483b
514d 3924 6b24
! - t sp esc $ B $ H ; H M
Q $ 9 $ k
! 0010040 4824 2d24 4b21 5e24 3f24
4f24 3d49 283c
! $ H $ - ! K $ ^ $ ? $
O I = < (

(2) euc-jp to iso2022-jp read-puts conversion

When I convert a text with "tclsh8.4 eucjis.tcl -eucjis
-read infile outfile",
sometimes extra "esc $ B" appears in the middle of the
output.
It seems it always appears at around the character
number 4096 or
8192, etc. (It's not byte number, but character
number.) Thus,
if the tcl internal buffer for unicode storage is
8192-byte long
(4096 characters), such boundary handling is supposed
to have some
bugs, at the beginning of each internal buffer.

(3) font selection mechanism

Under tk8.4a4 some character is not displayed correctly
with a font
like "*-jisx0208.1983-1". It is a minor problem, since
we normally use
"*-jisx0208.1983-0".

>