#100 expIRead misinterprets "0 bytes read" as EOF

[heinrichmartin]

Expect version: 5.45 (src.rpm from fc22 backported to rhel65)
Tcl version: 8.6.3 (src.rpm from fc22 backported to rhel65)

Unable to reproduce the behavior in a simplified setup, but here is what happens: When the syscall read() returns EAGAIN, then expIRead returns 0 which is documented to be EOF.

Problem Behaviour:

• Tcl's DoReadChars returns 0 (= zero chars read) with errno set to EAGAIN (11). This behavior is unchanges from the beginning of this function.
• expIRead does nothing in this case (cc = 0) and returns 0. The return value 0 is documented to report EOF.
• expRead then conclusively returns EXP_EOF (-11) to Exp_ExpectObjCmd.
• The eof pattern of the calling [expect] is matched.

Expected Behavior:
The calling [expect] times out gracefully, i.e. the timeout pattern matches.

[Don Porter]

The I/O subsystem released in Tcl 8.6.3 is broken. You should not use that release. Upgrade to Tcl 8.6.4 and see if the problem continues.

[heinrichmartin]

The problem continues after the upgrade. Here are more facts:

• EAGAIN == EWOULDBLOCK originates from syscall read().
• The call stack at read() with 8.6.3 was:
  ++ ExpInputProc
  ++ ChanRead
  ++ GetInput
  ++ DoReadChars
  ++ expIRead
  ++ expRead
  ++ Exp_ExpectObjCmd
• The following conditions are required (but unfortunately not sufficient) to reproduce the bug in my setup:
  ++ -timeout 0
  ++ expect on a socket
  ++ expect with no (further) data available

I wondered whether Tcl or Expect is designed to only ever call read() when data is available and EWOULDBLOCK should not happen at all.

• If so, then this design is broken somewhere.
• If not, then one of the functions from the call stack makes EOF out of EWOULDBLOCK.

[Don Porter]

Please provide a script that demonstrates the problem.

Ideally you will confirm that the script works properly with
Tcl 8.6.1 but fails in the current release of Tcl 8.6.4.

Thank you.

[heinrichmartin]

Summary from private mail:

• can be reproduced with Tcl/Expect built from source (not only fc22 srpm)
• can be reproduced with Tcl 8.6.1 and 8.6.4
• can be found in the sources as follows (no public script / minimal example available)

When syscall read() returns -1 with EAGAIN or EWOULDBLOCK, then the call stack looks like this:

• read() - syscall
• ExpInputProc() - expect
• ChanRead() - resets the CHANNEL_EOF flag before the call; sets the CHANNEL_BLOCKED flag after the call; unifies EAGAIN || EWOULDBLOCK as EAGAIN
• GetInput()
• DoReadChars()
• Tcl_ReadChars() - returns 0, with errno EAGAIN ("If the channel is in nonblocking mode, a return value of zero indicates either that no input is currently available or an end-of-file condition. Use Tcl_Eof and Tcl_InputBlocked to tell which of these conditions actually occurred.")
• expIRead() - forwards the return value 0, but code comment says this means EOF
• expRead()
• Exp_ExpectObjCmd()

The attached patch resolves the problem for me. I chose to patch expIRead in order to keep the code comment's promise (to return 0 in case of EOF).

[heinrichmartin]

This is an alternative patch that is more simple, but it does not fix the misinterpretation in expIRead. Other bugs might be dormant and the patch is untested.

[heinrichmartin]

This is actually fixed in 5.45.3, but which is not the latest downloadable file.

https://sourceforge.net/p/expect/bugs/99/
https://sourceforge.net/p/expect/patches/18/ (superseds https://sourceforge.net/p/expect/patches/14/)

[Nils Carlson]

Fixed in 5.45.3 .