[ clisp-Bugs-1232806 ] win32 console copy&paste yields errors about 'L'

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Bugs item #1232806, was opened at 2005-07-05 10:58
Message generated for change (Comment added) made by sds
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=101355&aid=1232806&group_id=1355

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: clisp
Group: lisp error
Status: Open
Resolution: None
Priority: 5
Submitted By: Jörg Höhle (hoehle)
Assigned to: Sam Steingold (sds)
Summary: win32 console copy&paste yields errors about 'L'

Initial Comment:
Hi,

there have been mentions in the mailing list here and 
there, but no use of the bug tracker.

Using a DOS console under MS-Windows-2k, it can happen 
that CLISP complains about a spurious L, as in
*** - Invalid option in
       (FFI:DEF-CALL-OUT GETPID (:NAME 
"GetCurrentProcessId")
        (:LIBRARY "kernel32.dll") L (:RETURN-TYPE 
FFI:UINT32))
      : L

I just happened to me now, so I investigated the 
symptoms a little bit.
(defun foo() (loop for c = (read-char *standard-input* 
nil nil nil) while c until (char= c #\%) collect c))
(foo) (coerce * 'string)

"(ffi:def-call-out getpid (:name 
\GetCurrentProcessId\)
l (:library \kernel32.dll\)  (:return-type 
ffi:uint32))

It now appears that the spurious L is overwriting the 
last line of copy&paste. Here's a sample test:
(a
 b
 c
 d
 f)
-> "(a
 b
 c
 d
lf)
"
Further observations:
o The command line history of the MS-Windows console 
does not show the junk, it looks correct.
o It happens with Copy&Paste from Netscape or from 
Emacs-20.7.

o Work-around -- It does not happen when the copy&paste 
block ends at the start of a new line (instead of after 
the last character of things to copy). I had never 
heard of a work-around before!

Using clisp-cvs, built with MS-VC 6.0 and 
libsigsegv/cvs.
Regards,
 Jörg Höhle.

----------------------------------------------------------------------

>Comment By: Sam Steingold (sds)
Date: 2005-07-08 15:21

Message:
Logged In: YES 
user_id=5735

BTW, Jorg, could you please add all woe32 FFI forms you
write to clisp/modules/bindings/win32/win32.lisp?
let up grow the module!
Thanks!

----------------------------------------------------------------------

Comment By: Sam Steingold (sds)
Date: 2005-07-07 13:04

Message:
Logged In: YES 
user_id=5735

I confirm.
[
you also need
(def-c-type handle c-pointer)
(def-c-type dword uint32)
]

I suggest that you get this information to CERT ASAP.
we also need to switch to buffered consile i/o.
terminal2 is no longer used and ChangeLog offers no
explanations.
maybe you could try enabling it in woe32?

----------------------------------------------------------------------

Comment By: Jörg Höhle (hoehle)
Date: 2005-07-07 12:01

Message:
Logged In: YES 
user_id=377168

The bug appears when doing 1 character/byte at a time I/O to 
the MS-Windows console. Thus obviously, it's a long-standing 
bug in MS-Windows. However CLISP must question itself 
whether 1 by 1 I/O is that great... (It remembers me how it 
killed performance on AmigaOS).

I tried to mimic CLISP's stream.d/win32aux.d using the FFI
(def-c-type sword sint32)
(def-call-out GetStdHandle (:name"GetStdHandle")(:library 
"kernel32.dll")
  (:return-type handle) (:language :stdc-stdcall)
  (:arguments (stdhandle sword)))
(setq conin (GetStdHandle -10)) ; input handle
http://msdn.microsoft.com/library/default.asp?url=/library/e
n-us/dllproc/base/getstdhandle.asp

(setq buf (allocate-shallow 'character :count 400))
(def-call-out ReadFile (:name"ReadFile")(:library 
"kernel32.dll")
  (:return-type boolean) (:language :stdc-stdcall)
  (:arguments (handle handle) (buffer c-pointer)
	      (bytestoread dword) (bytesread (c-ptr dword) 
:out)
	      (overlapped c-pointer)))
(defun foo1 ()
  (loop do (multiple-value-setq (ok num) (ReadFile conin buf 
1 nil))
        while ok do
        (prin1 (setq read (foreign-value(foreign-variable 
buf(parse-c-type `(c-array uint8 ,num))))))
        until (position (char-code #\%) read)))

#(108)#(32)#(32)#(32)#(32)#(32)#(32)#(32)#(119)#(104)#(105)#
(108)#(101)#(32)#(111)#(107)#(32)#(100)#(111)#(13)#(10)%

Note that a priori, 1by1 is a waste for the console, since 
ReadFile() returns for each Return (even when using 
copy&paste), at least on my MS-Windows-2k box.
So a line-buffered approach to console I/O would be a much 
better approach.
I vaguely remember that the AmigaCLISP used a line buffer 
also?? or was that one of the three HAVE_TERMINAL1/2/3 which 
did this? I could't find HAVE_TERMINAL2 anymore in stream.d

Experimenting a little more, reading 5 by 5, instead of #\l 
I get 5 completely junk characters
#(108 0 105 0 115)#(32 32 32 119 104)
overwriting 5 spaces from Copy&Paste... Looks like UNICODE 
junk from somewhere -- with an even longer buffer, it says 
"lisp.exe"(!)
Now, how can one get MS to fix this??
Have my colleagues create a CERT buffer overflow alert??

The bug even showed up reading by blocks of 30 characters. 
But not with 100. It's probably silent when the buffer can 
hold a whole line.

Regards,
 Jörg

----------------------------------------------------------------------

Comment By: Jörg Höhle (hoehle)
Date: 2005-07-07 11:33

Message:
Logged In: YES 
user_id=377168

The bug does not appear in RAW mode, i.e.
(with-keyboard (loop (print(read-char *keyboard-input*))))

----------------------------------------------------------------------

Comment By: Jörg Höhle (hoehle)
Date: 2005-07-06 05:37

Message:
Logged In: YES 
user_id=377168

Thanks for the reminder, I had forgotten those 
particular e-mails.

Sam wrote:
>OK, so the symptom is:
>in a clisp build with mingw
>when we paste a multi-line expression into a console
>and the expression does _NOT_ end with a newline,
>the last newline (and only it?) is read by CLISP as #\l (or 
#\L ?)

I'd phrase it slightly differently:
With either a mingw build, using a MS-XP console,
or an MS-VC-6.0 build, using a MS-2k console,
the first character of the last line is overwritten with #\l 
(and not #\L as my read-char tests shows) -- unless the line 
is terminated by a newline.

I don't know how to choose among pasting CR or CRLF or LF 
terminated text into the console. I don't know whether the 
clipboard transforms line-terminators. I just select text 
from another MS-Windows GUI app and paste it.

It still does not rule out a bug in CLISP, instead of 
MS-windows consoles IMHO.

----------------------------------------------------------------------

Comment By: Sam Steingold (sds)
Date: 2005-07-05 11:19

Message:
Logged In: YES 
user_id=5735

https://sourceforge.net/mailarchive/message.php?msg_id=11257167
https://sourceforge.net/mailarchive/message.php?msg_id=11862707
https://sourceforge.net/mailarchive/message.php?msg_id=11868691
https://sourceforge.net/mailarchive/message.php?msg_id=11887553
https://sourceforge.net/mailarchive/message.php?msg_id=11892871

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=101355&aid=1232806&group_id=1355