[Tcl-bugs] [ tcl-Bugs-3436609 ] Transforms/Encodings may generate false readable fileevent

[SourceForge.net <no...@so...>]

Bugs item #3436609, was opened at 2011-11-11 09:29
Message generated for change (Tracker Item Submitted) made by ferrieux
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110894&aid=3436609&group_id=10894

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: 25. Channel System
Group: None
Status: Open
Resolution: None
Priority: 8
Private: No
Submitted By: Alexandre Ferrieux (ferrieux)
Assigned to: Andreas Kupries (andreas_kupries)
Summary: Transforms/Encodings may generate false readable fileevent

Initial Comment:
Channel transforms and (multibyte) encodings break the byte/char 1-to-1 mapping. As a consequence, there are cases where the low-level (resp. raw) channel has a byte ready (as detected by the select() in the notifier, letting a [fileevent readable] fire), but a [read 1] on the hi-level (resp. decoded) channel would block (if blocking) or return the empty string (if not).

The possibility of such "false positives" in readable fileevents is not advertised, and a recurring idiom is

   fconfigure $ch -blocking 0
   fileevent $ch readable foo
   proc foo {} {
      set data [read $::ch 1000]
      if {![string length $data]} {close $::ch;clean_things_up}
      # use $data
    }

This idiom, of course, leads to erroneous truncation. The same thing at C level leads to 2762041 and 1945538.
Note that this mechanism potentially applies to all multibyte encodings and channel transforms. TLS and Zlib are highly suspect of suffering from this, along with the ubiquitous [-encoding utf-8], as shown by the attached script.

Two options:

 (1) fix it the hard and clean way, by doing a read-ahead inside the NotifyProc (see 1945538 for details). Do this in the core for non-single-byte encodings, and in the transform-based extensions.

 (2) work around it, by strongly documenting the fact. Possibly many scripts to fix. Still some C code to update in extensions making the wrong assumption, to stop them from giving false EOFs.



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110894&aid=3436609&group_id=10894