From: edgar <edg...@we...> - 2010-04-18 05:31:22
|
Bug #2795278 - Terminal kill under Debian leads to hungry zombie Hello CLISP list, Sam Steingold wrote on clisp-devel: > I cannot reproduce this. > If you can, please compile clisp with debugging symbols > (http://clisp.podval.org/impnotes/faq.html#faq-debug) > and attach gdb to the run-away clisp and send us the backtrace. > if you cannot recompile clisp, please use strace and ltrace to > figure out what clisp is doing. I'm not the original bug reporter, but I encountered the same problem several times in the past on Ubuntu (based on Debian), and now I have here a machine running Ubuntu_9.10 with Gnome, where I can reliably reproduce the bug. It happens *every* time after I close a terminal window without quitting CLISP before. The zombie only remains if I start CLISP via the 'clisp' loader: bash$ clisp bash$ clisp -K base bash$ clisp -K full and only if I close the xterm window via the Gnome 'close' button without quitting CLISP before. Only then a zombie remains. Maybe important: NO Zombie remains if I start CLISP directly via: bash$ /usr/local/lib/clisp-2.48/base/lisp.run \ -B /usr/local/lib/clisp-2.48 \ -M /usr/local/lib/clisp-2.48/base/lispinit.mem \ -N /usr/local/share/locale bash$ /usr/local/lib/clisp-2.48/full/lisp.run \ -B /usr/local/lib/clisp-2.48 \ -M /usr/local/lib/clisp-2.48/full/lispinit.mem \ -N /usr/local/share/locale Beause I'm a hardware electrician and not a software scientist, here is a detailed description what I have done so far. I have read: http://clisp.podval.org/impnotes/faq.html#faq-debug then I have compiled clisp-2.48 with debugging symbols: --- start of configure transcript --- bash$ ./configure --with-debug ubuntu-build Configure findings: FFI: yes (user requested: default) readline: yes (user requested: default) libsigsegv: yes ./makemake --with-dynamic-ffi --verbose=yes --with-libsigsegv-prefix=/ home/edgar/Downloads/clisp/clisp-2.48/tools/i686-pc-linux-gnu --srcdir= ../src debug > Makefile with_libsigsegv_prefix=/home/edgar/Downloads/clisp/clisp-2.48/tools/i68 6-pc-linux-gnu # host system: hostname = "compaq-evo" HSYS = "i686" HSYSOS = "linux" HOS = "unix" host_cpu = "i686" cpu = "i386" host_os = "linux-gnu" host = "i686-pc-linux-gnu" # target system: TSYS = "i686" TSYSOS = "linux" TOS = "unix" --- end of configure transcript --- After "make", "make check" (no errors) and "make install", I did: bash$ clisp -K full Then I closed the xterm window via the Gnome 'close' button and a CLISP zombie remained, as I can see from a second xterm window: bash$ top PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 19242 edgar 20 0 19364 14m 2044 R 98.4 3.0 0:11.71 lisp.run bash$ ps ax | grep clisp 19242 ? R 0:22 /usr/local/lib/clisp-2.48/full/lisp.run -B /usr/local/lib/clisp-2.48 -M /usr/local/lib/clisp-2.48/full/lispinit.m em -N /usr/local/share/locale -K full From the second xterm window I attached GDB to the CLISP zombie: bash$ gdb clisp 19242 --- start of gdb transcript --- GNU gdb (GDB) 7.0-ubuntu Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i486-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/local/bin/clisp...done. Attaching to program: /usr/local/bin/clisp, process 19242 A program is being debugged already. Kill it? (y or n) [typed: n] (gdb) bt #0 0x08090c4c in ?? () #1 0x0809597f in ?? () #2 0x080926c3 in ?? () #3 0x08090c91 in ?? () #4 0x08095a3f in ?? () #5 0x080926c3 in ?? () #6 0x08090c07 in ?? () #7 0x08094170 in ?? () #8 0x080926c3 in ?? () #9 0x08090c91 in ?? () #10 0x0809597f in ?? () ... --- end of gdb transcript --- *** PROBLEM: my first mail came back with: Bounce: The message's content type was not explicitly allowed The problem probably is that the "gdb.log.gz" file I had sent with the original mail was gz-compressed. But the backtrace ist approx. 20000 lines long, so the uncompressed file has a size of 550kB ... I'm not sure if this really is the backtrace you wanted to have, so in case it's not, I would need some advice from the software gurus on this list how to produce better bug reports or some way how to send the file as a compressed attachment. I'm not in hurry, I do not need this machine for the next four weeks. I would be willing to repeat the whole stuff with a CVS HEAD version if this would make more sense. Thanks, - edgar -- The author of this email does not necessarily endorse the following advertisements, which are the sole responsibility of the advertiser: |
From: Sam S. <sd...@gn...> - 2010-04-18 14:55:11
|
> * edgar <rqtne-esg@jro.qr> [2010-04-18 07:30:58 +0200]: > > I'm not in hurry, I do not need this machine for the next four weeks. > I would be willing to repeat the whole stuff with a CVS HEAD version > if this would make more sense. the bug should be fixed in the cvs head. -- Sam Steingold (http://sds.podval.org/) on Ubuntu 9.10 (karmic) http://thereligionofpeace.com http://mideasttruth.com http://pmw.org.il http://honestreporting.com http://openvotingconsortium.org http://dhimmi.com A bullet affects the way the brain functions even when it hits the butt. |
From: edgar <edg...@we...> - 2010-04-19 02:05:45
|
Bug #2795278 - Terminal kill under Debian leads to hungry zombie Sam Steingold wrote: > the bug should be fixed in the cvs head. Confirmation: the bug does not appear any more with CLISP cvs head. And yes, I should have tried this before. Sorry for the confusion. In case anybody is interested in the following: --- make check with CLISP CVS HEAD (April 18, 2010) on Ubuntu_9.10 --- 9 eval20: 1 error out of 44 tests bash$ cat tests/*.erg Form: (WITH-OUTPUT-TO-STRING (*ERROR-OUTPUT*) (DECLAIM (OPTIMIZE ZOT))) CORRECT: "WARNING: ZOT is not a valid OPTIMIZE quality. " CLISP : "WARNUNG: ZOT is not a valid OPTIMIZE quality. " Differ at position 4: #\I vs #\U CORRECT: "ING: ZOT i..." CLISP : "UNG: ZOT i..." Thanks, - edgar |
From: Sam S. <sd...@gn...> - 2010-04-19 22:09:11
|
edgar wrote: > > In case anybody is interested in the following: > > --- make check with CLISP CVS HEAD (April 18, 2010) on Ubuntu_9.10 --- > > 9 eval20: 1 error out of 44 tests > > bash$ cat tests/*.erg > > Form: (WITH-OUTPUT-TO-STRING (*ERROR-OUTPUT*) (DECLAIM (OPTIMIZE ZOT))) > CORRECT: "WARNING: ZOT is not a valid OPTIMIZE quality. > " > CLISP : "WARNUNG: ZOT is not a valid OPTIMIZE quality. > " > Differ at position 4: #\I vs #\U > CORRECT: "ING: ZOT i..." > CLISP : "UNG: ZOT i..." please try this patch: --- Makefile.~1.21.~ 2008-11-20 15:23:48.000000000 -0500 +++ Makefile 2010-04-19 18:08:01.003211000 -0400 @@ -6,7 +6,7 @@ BD=../ # executable extension (.exe on win32, .run everywhere else) LEXE=.run -LISP=$(BD)lisp$(LEXE) -E utf-8 -norc -B $(BD) -N $(BD)locale -M $(BD)lispinit.mem -m 30MW +LISP=$(BD)lisp$(LEXE) -E utf-8 -norc -B $(BD) -N $(BD)locale -M $(BD)lispinit.mem -m 30MW -L english # LISP=../clisp -norc # LISP=clisp -norc |
From: edgar <edg...@we...> - 2010-04-20 11:15:32
|
>>edgar wrote: >> >> In case anybody is interested in the following: >> >> --- make check with CLISP CVS HEAD (April 18, 2010) on Ubuntu_9.10 --- >> >> 9 eval20: 1 error out of 44 tests >> >> bash$ cat tests/*.erg >> >> Form: (WITH-OUTPUT-TO-STRING (*ERROR-OUTPUT*) (DECLAIM (OPTIMIZE ZOT))) >> CORRECT: "WARNING: ZOT is not a valid OPTIMIZE quality. >> " >> CLISP : "WARNUNG: ZOT is not a valid OPTIMIZE quality. >> " >> Differ at position 4: #\I vs #\U >> CORRECT: "ING: ZOT i..." >> CLISP : "UNG: ZOT i..." Sam Steingold wrote: >please try this patch: > >--- Makefile.~1.21.~ 2008-11-20 15:23:48.000000000 -0500 >+++ Makefile 2010-04-19 18:08:01.003211000 -0400 >@@ -6,7 +6,7 @@ BD=../ > # executable extension (.exe on win32, .run everywhere else) > LEXE=.run > >-LISP=$(BD)lisp$(LEXE) -E utf-8 -norc -B $(BD) -N $(BD)locale -M $(BD)lispinit.mem -m 30MW >+LISP=$(BD)lisp$(LEXE) -E utf-8 -norc -B $(BD) -N $(BD)locale -M $(BD)lispinit.mem -m 30MW -L english > # LISP=../clisp -norc > # LISP=clisp -norc Thanks, I did, but unfortunately nothing changed. Then I did a new CVS checkout, applied the patch, and did a complete "./configure", "make", "make check", but still the same result: ./configure --with-libsigsegv-prefix=${prefix} ubuntu-build make make check --- make check with CLISP CVS HEAD (April 20, 2010) on Ubuntu_9.10 --- 9 eval20: 1 error out of 44 tests bash$ cat tests/*.erg Form: (WITH-OUTPUT-TO-STRING (*ERROR-OUTPUT*) (DECLAIM (OPTIMIZE ZOT))) CORRECT: "WARNING: ZOT is not a valid OPTIMIZE quality. " CLISP : "WARNUNG: ZOT is not a valid OPTIMIZE quality. " Differ at position 4: #\I vs #\U CORRECT: "ING: ZOT i..." CLISP : "UNG: ZOT i..." Any other things I can do? - edgar |
From: edgar <edg...@we...> - 2010-04-20 15:42:01
|
Localization problems (was: Terminal zombie bug...) I have changed the title because this has nothing to do anymore with the terminal bug. > --- make check with CLISP CVS HEAD (April 20, 2010) on Ubuntu_9.10 --- > > 9 eval20: 1 error out of 44 tests > > bash$ cat tests/*.erg > > Form: (WITH-OUTPUT-TO-STRING (*ERROR-OUTPUT*) (DECLAIM (OPTIMIZE ZOT))) > CORRECT: "WARNING: ZOT is not a valid OPTIMIZE quality. > " > CLISP : "WARNUNG: ZOT is not a valid OPTIMIZE quality. > " > Differ at position 4: #\I vs #\U > CORRECT: "ING: ZOT i..." > CLISP : "UNG: ZOT i..." The "ING" (english) vs. "UNG" (german) error is based on the fact that "make test" expects English error messages while CLISP produces German error messages, even if CUSTOM:*CURRENT-LANGUAGE* is set to ENGLISH. So I did some further investigations... If I start CLISP CVS HEAD via: bash$ clisp -L english then, inside CLISP, I get: [1]> CUSTOM:*CURRENT-LANGUAGE* ENGLISH but all error messages are still in German language (as far as translated at all), this is the problem. If I start CLISP CVS HEAD via LANG=EN like this: bash$ LANG=EN clisp bash$ LANG=EN clisp -L english then, with or without -L english, inside CLISP I get the following error messages at start-up: ** - Continuable Error DIRECTORY: Invalid byte #xC3 in CHARSET:ASCII conversion If you continue (by typing 'continue'): Discard this directory entry The following restarts are also available: STORE-VALUE :R1 Input a new value for *PATHNAME-ENCODING*. Break 1 [3] The problem probably is that the Englisch encoding does not understand German umlauts (at least this is a very common problem in Germany). Only if I start CLISP CVS HEAD via LC_MESSAGES=EN like this: bash$ LC_MESSAGES=EN clisp bash$ LC_MESSAGES=EN clisp -L english then, with or without -L english, inside CLISP I get: [1]> CUSTOM:*CURRENT-LANGUAGE* ENGLISH and now also all error messages are in English language. Does the i18n cod look-up LC_MESSAGES at runtime? I have found the i18n code in "modules/i18n", but still had no closer look at it. Is there any other i18n code to take into account? I will try to fiddle out what's going on there later on... Have a nice day, - edgar -- The author of this email does not necessarily endorse the following advertisements, which are the sole responsibility of the advertiser: |
From: Sam S. <sd...@gn...> - 2010-04-20 18:09:06
|
edgar wrote: > > > --- make check with CLISP CVS HEAD (April 20, 2010) on Ubuntu_9.10 --- > > > > 9 eval20: 1 error out of 44 tests > > > > bash$ cat tests/*.erg > > > > Form: (WITH-OUTPUT-TO-STRING (*ERROR-OUTPUT*) (DECLAIM (OPTIMIZE ZOT))) > > CORRECT: "WARNING: ZOT is not a valid OPTIMIZE quality. > > " > > CLISP : "WARNUNG: ZOT is not a valid OPTIMIZE quality. > > " > > Differ at position 4: #\I vs #\U > > CORRECT: "ING: ZOT i..." > > CLISP : "UNG: ZOT i..." > > The "ING" (english) vs. "UNG" (german) error is based on the > fact that "make test" expects English error messages while CLISP > produces German error messages, even if CUSTOM:*CURRENT-LANGUAGE* > is set to ENGLISH. > > So I did some further investigations... > > If I start CLISP CVS HEAD via: > > bash$ clisp -L english > > then, inside CLISP, I get: > > [1]> CUSTOM:*CURRENT-LANGUAGE* > ENGLISH > > but all error messages are still in German language > (as far as translated at all), this is the problem. > > If I start CLISP CVS HEAD via LANG=EN like this: > > bash$ LANG=EN clisp > bash$ LANG=EN clisp -L english > > then, with or without -L english, inside CLISP I get > the following error messages at start-up: > > ** - Continuable Error > DIRECTORY: Invalid byte #xC3 in CHARSET:ASCII conversion > If you continue (by typing 'continue'): Discard this directory entry > The following restarts are also available: > STORE-VALUE :R1 Input a new value for *PATHNAME-ENCODING*. > Break 1 [3] > > The problem probably is that the Englisch encoding does not understand > German umlauts (at least this is a very common problem in Germany). > > Only if I start CLISP CVS HEAD via LC_MESSAGES=EN like this: > > bash$ LC_MESSAGES=EN clisp > bash$ LC_MESSAGES=EN clisp -L english > > then, with or without -L english, inside CLISP I get: > > [1]> CUSTOM:*CURRENT-LANGUAGE* > ENGLISH > > and now also all error messages are in English language. > > Does the i18n cod look-up LC_MESSAGES at runtime? looks like it: $ LC_MESSAGES=de_DE ./clisp -q -norc -K boot -x 'custom:*current-language* (warn"foo")' ENGLISH WARNUNG: foo NIL $ $ LC_MESSAGES=en_UK ./clisp -q -norc -K boot -L german -x 'custom:*current-language* (warn"foo")' DEUTSCH WARNUNG: foo NIL $ this is, indeed, confusing... Bruno, could you please explain what you mean here? > I have found the i18n code in "modules/i18n", but still had no closer > look at it. Is there any other i18n code to take into account? the code in modules/i18n/ is for user programs, see http://clisp.cons.org/impnotes/i18n-mod.html what you are interested in is documented in http://clisp.cons.org/impnotes/i18n.html http://clisp.cons.org/impnotes/clisp.html#opt-lang the main place where to look at is clisp/src/spvw_language.d:init_language() it is fairly involved though, and uses a lot of "#if"s. (you might want to do "make lispbibl.h" in your build directory to collect all definitions in one file for your examination). Sam. |
From: Sam S. <sd...@gn...> - 2010-04-20 19:22:38
|
edgar wrote: > > CORRECT: "WARNING: ZOT is not a valid OPTIMIZE quality. > > " > > CLISP : "WARNUNG: ZOT is not a valid OPTIMIZE quality. > > " looks like LC_MESSAGES=en_US ./clisp -L english is the way. please try again after a "cvs up" thanks! |
From: edgar <edg...@we...> - 2010-04-21 05:13:13
|
edgar: > I have found the i18n code in "modules/i18n"... > Is there any other i18n code to take into account? Sam Steingold: > the main place where to look at is > clisp/src/spvw_language.d:init_language() After reading "clisp/src/spvw_language.d" I realized that some of the asssumed problems had been caused by me myself. The correct gettext 'language' argument must be given in in lowercase, not in uppercase like I did. The second, uppercase part, defines the charset to use, not the language for the translation. So I tried to change the "tests/Makefile" into something like this: LISP=LANG=en_US $(BD)lisp$(LEXE) -E utf-8 -norc -B $(BD) -N $(BD)locale -M $(BD)lispinit.mem -m 30MW But this was no good idea because 'make check' then starts to re-compile all locale-dependent parts of CLISP (regexp, readline, etc.) from German into English versions. :( After further reading the CLISP i18n documentation I had a look at my (unmodified) Ubuntu language settings: bash$ LANG => de_DE.UTF-8 bash$ LC_MESSAGES => "" (nothing) bash$ LC_ALL => "" (nothing) bash$ LC_CTYPE => "" (nothing) After reading "de_DE.UTF-8" I did a probably important observation: $ LANG=de_DE ./clisp -q -norc -K boot -x 'custom:*current-language* (warn"foo")' ENGLISH WARNING: foo NIL $ LC_LANG=de_DE.UTF-8 ./clisp -q -norc -K boot -x 'custom:*current-language* (warn"foo")' ENGLISH WARNUNG: foo NIL ... "de_DE" results in WARNING (english) ... "de_DE.UTF-8" results in WARNUNG (german) Maybe this behaviour has changed in recent 'gettext' versions? The same effect can be observed with LC_MESSAGES: $ LC_MESSAGES=de_DE ./clisp -q -norc -K boot -x 'custom:*current-language* (warn"foo")' ENGLISH WARNING: foo NIL $ LC_MESSAGES=de_DE.UTF-8 ./clisp -q -norc -K boot -x 'custom:*current-language* (warn"foo")' ENGLISH WARNING: foo NIL "de_DE" results in WARNING, "de_DE.UTF-8" results in WARNUNG. Please note that in both cases the CUSTOM:*CURRENT_LANGUAGE* was set to the ENGLISH value. I'm not sure if this is a bug at all, it could be possible that CUSTOM:*CURRENT_LANGUAGE* only defines the language for the programmable translations via the "i18n" module and therefore explicitely must be set via the "-L language" command line option or via the CLISP_LANGUAGE environment variable? After this I did the next experiment: LANG=de_DE ./configure --with-libsigsegv-prefix=${prefix} ubuntu-build LANG=de_DE make LANG=de_DE make check With "LANG=de_DE" instead of the Ubuntu "LANG=de_DE.UTF-8" I get no more error messages during "make check". Obviously CLISP doesn't like the Ubuntu .UTF-8 extension. Unfortunately I'm not an experienced-enough C programmer to fix this myself, but I would be willing to do further experiments and source code investigations if anybody gives me a hint where to search. Thanks, - edgar -- The author of this email does not necessarily endorse the following advertisements, which are the sole responsibility of the advertiser: |
From: edgar <edg...@we...> - 2010-04-21 14:44:08
|
> clisp-cvs Digest, Vol 48, issue 6: > clisp/src ChangeLog,1.7318,1.7319 spvw_language.d,1.42,1.43 Yes, thanks a lot, works better, no "make check" errors any more. Just tested with Ubuntu 9_10 and LANG=de_DE.UTF-8: bash$ clisp This produces German messages. bash$ LANG=en_US.UTF-8 clisp This produces English messages (important for the mailing lists). But unfortunately the following lines still don't work: bash$ LANG=de clisp bash$ LANG=de_DE clisp bash$ LANG=en clisp bash$ LANG=en_EN clisp all lines above produce at start-up the same error: ** - Continuable Error DIRECTORY: Invalid byte #xC3 in CHARSET:ASCII conversion If you continue (by typing 'continue'): Discard this directory entry The following restarts are also available: STORE-VALUE :R1 Input a new value for *PATHNAME-ENCODING*. Break 1 [3]> The "#xC3" is the German umlaut "AE" (a big A with two dots above). It looks as if CLISP now needs the explicit .UTF-8 encoding, otherwise not even a German locale works in a German environment. Before you get nuts with it: I already have read half of the gettext manual and will look at it again later on... Thanks, - edgar -- The author of this email does not necessarily endorse the following advertisements, which are the sole responsibility of the advertiser: |
From: Sam S. <sd...@gn...> - 2010-04-21 15:21:42
|
edgar wrote: > > bash$ LANG=de clisp > bash$ LANG=de_DE clisp > bash$ LANG=en clisp > bash$ LANG=en_EN clisp spvw_language.d says The analysis of getenv("LANG") below will be done - in more detail - by bindtextdomain() and textdomain(). No need to do it ourselves. i.e., LANG is not processed by CLISP. > all lines above produce at start-up the same error: > > ** - Continuable Error > DIRECTORY: Invalid byte #xC3 in CHARSET:ASCII conversion > If you continue (by typing 'continue'): Discard this directory entry > The following restarts are also available: > STORE-VALUE :R1 Input a new value for *PATHNAME-ENCODING*. > Break 1 [3]> http://clisp.cons.org/impnotes/faq.html#faq-enc-err http://clisp.cons.org/impnotes/faq.html#faq-fine |
From: edgar <edg...@we...> - 2010-04-27 16:22:41
|
keyboard stream produces nonsense with non-ascii characters? *terminal-io* works ok, but ext:*keyboard-input* produces nonsense German umlaut example: > (read-char) ö <-umlaut oe typed by me #\LATIN_SMALL_LETTER_O_WITH_DIAERESIS ; correct result > (ext:with-keyboard (read-char ext:*keyboard-input*)) ; I typed again the umlaut oe #S(SYSTEM::INPUT-CHARACTER :CHAR #\LATIN_CAPITAL_LETTER_A_WITH_TILDE :BITS 0 :FONT 0 :KEY NIL) ; nonsense result There is no #\LATIN_CAPITAL_LETTER_A_WITH_TILDE umlaut in Germany. Exactly the same nonsense result I get with *all* German umlauts. I assume the reason is that in UTF-8 the German umlaut "oe" is encoded as the two-byte sequence #xc3 #xf6 because if I type: > (code-char #xc3) #\LATIN_CAPITAL_LETTER_A_WITH_TILDE In UTF-8 *all* German two-byte umlaut sequences start with #xc3, and that's why I always get the wrong #\LATIN_CAPITAL_LETTER_A_WITH_TILDE Similar problems arise with non-ascii characters (e.g. arrow-keys) together with the Ctrl, Alt, or Shift keys. In most cases I get Escape-sequences completely different than the Escape-sequences defined in "clisp/src/stream.d". (I can provide detailed examples if needed.) By looking at "clisp/src/stream.d" it looks to me as if the keyboard-stream code still uses the old ASCII encoding. The question is: Was the keyboard-stream code just simply never updated since the Unicode support was integrated into CLISP or is there any additional Linux setup-stuff I have to do on my system? (Is there a simple answer possible at all?) Thanks, - edgar -- The author of this email does not necessarily endorse the following advertisements, which are the sole responsibility of the advertiser: |
From: Sam S. <sd...@gn...> - 2010-04-27 17:29:07
|
edgar wrote: > keyboard stream produces nonsense with non-ascii characters? > > *terminal-io* works ok, but ext:*keyboard-input* produces nonsense well, actually, as you discern yourself below, it merely returns bytes instead of characters. > German umlaut example: > > > (read-char) > ö <-umlaut oe typed by me > #\LATIN_SMALL_LETTER_O_WITH_DIAERESIS ; correct result > > > (ext:with-keyboard (read-char ext:*keyboard-input*)) > ; I typed again the umlaut oe > #S(SYSTEM::INPUT-CHARACTER :CHAR #\LATIN_CAPITAL_LETTER_A_WITH_TILDE > :BITS 0 :FONT 0 :KEY NIL) ; nonsense result > > There is no #\LATIN_CAPITAL_LETTER_A_WITH_TILDE umlaut in Germany. > > Exactly the same nonsense result I get with *all* German umlauts. > I assume the reason is that in UTF-8 the German umlaut "oe" is encoded > as the two-byte sequence #xc3 #xf6 because if I type: > > > (code-char #xc3) > #\LATIN_CAPITAL_LETTER_A_WITH_TILDE > > In UTF-8 *all* German two-byte umlaut sequences start with #xc3, and > that's why I always get the wrong #\LATIN_CAPITAL_LETTER_A_WITH_TILDE > > Similar problems arise with non-ascii characters (e.g. arrow-keys) > together with the Ctrl, Alt, or Shift keys. In most cases I get > Escape-sequences completely different than the Escape-sequences > defined in "clisp/src/stream.d". > > By looking at "clisp/src/stream.d" it looks to me as if the > keyboard-stream code still uses the old ASCII encoding. Indeed, the with-keyboard facility predates unicode in clisp by at least 6 years. The same, btw, goes for screen. Search for /* FIXME: This should take into account the encoding. */ in stream.d and you will see that, indeed, the encodings of window and keyboard streams (which are initialized to *terminal-encoding*) are ignored. Of course, http://www.cygwin.com/acronyms/#PTC . Sam. PS. please start a new thread with a new topic, i.e., do not use "reply" when starting a new discussion. this message was filed under your previous message by thunderbird :-( |
From: edgar <edg...@we...> - 2010-04-27 19:10:07
|
Re: keyboard stream produces nonsense with non-ascii characters? > Indeed, the with-keyboard facility predates unicode in clisp by > at least 6 years. The same, btw, goes for screen. > > Search for /* FIXME: This should take into account the encoding. */ > in stream.d and you will see that, indeed, the encodings of window > and keyboard streams (which are initialized to *terminal-encoding*) > are ignored. > > Of course, http://www.cygwin.com/acronyms/#PTC. Thanks, I didn't want to nerd, the main thing I wanted to know is wether I am too stupid to set-up my own Linux system. As some kind of "work-around" I have discovered that I can read two-byte sequences by: (ext:with-keyboard (let ((input-char-1 (read-char ext:*keyboard-input*)) (input-char-2 (read-char ext:*keyboard-input*))) (list input-char-1 input-char-2))) but then I ran instantly into the next problems because either (read-char ext:*keyboard-input*) seems not to remove the input-char structure from the stream or peek-char has some bugs because after (read-char ext:*keyboard-input*) (peek-char ext:*keyboard-input*) peek-char still finds the old structure on the stream which was read by read-char before so I cannot use peek-char to detect if there are still characters to read, while read-char-no-hang doesn't recognize ext:*keyboard-input* char-structures at all. But this all is not too important, I ran across it because I was unable to implement arrow-key control with the screen package but now I know that both packages are from stone-age. I will drop this topic for now, but will look out for: /* FIXME: This should take into account the encoding. */ in the CLISP code in the future. > please do not use "reply" when starting a new discussion... Yes, I can confirm that with my Thunderbird it's exactly the same mess, that's why I had switch off 'sort by topic' long time ago... But I will try to remember to always start a fresh mail with every new topic. Thanks, - edgar -- The author of this email does not necessarily endorse the following advertisements, which are the sole responsibility of the advertiser: |
From: edgar <edg...@we...> - 2010-04-21 16:45:03
|
>> bash$ LANG=de clisp >> bash$ LANG=de_DE clisp >> bash$ LANG=en clisp >> bash$ LANG=en_EN clisp >> >> produce at start-up the error: >> >> ** - Continuable Error >> DIRECTORY: Invalid byte #xC3 in CHARSET:ASCII conversion >> If you continue (by typing 'continue'): Discard this directory entry >> The following restarts are also available: >> STORE-VALUE :R1 Input a new value for *PATHNAME-ENCODING*. >> Break 1 [3]> > > spvw_language.d says > > The analysis of getenv("LANG") below will be done - in more detail - > by bindtextdomain() and textdomain(). No need to do it ourselves. > > i.e., LANG is not processed by CLISP. Yes, I know, to me this looks not like a CLISP bug but more like a gettext or gllib problem. What I really want to know is what happens if I write some code e.g. under Windows under de_DE, ISO-8859-1 and \r\n line endings and then try to read it e.g. with CLISP on Linux with de_DE.UTF-8. The German umlauts will probably be scrambled because ISO-8859-1 is not UTF-8. That's normal and is a simple ENCODING issue, not a LOCALE issue. But as far as I have seen the ENCODING is derived from the LOCALE in clisp/src/gllib/config.charset (part of the unicode gllib) during ./configure, while the LOCALE is detected by gettext (not by CLISP), so maybe this could be the reason for this quirk? I will test this later on a Windows machine. The reason is that at work I only have windows machines, while at home I'm working on Linux. Copying source code back and forth between Windows and Linux is what I'm doing every day. (I also still have an old Mac around...) Please do not take all this too serious: I have compiled CLISP under de_DE.UTF-8 and CLISP recognizes correctly de_DE.UTF-8 and en_US.UTF-8. This is what the casual user expects, and this works now (thanks again). Everything else are typical multiplatform quirks. Multiplatform software always needs to make lots of compromises, so I'm afraid I will have to live with that. I only want to find out what I have to care about when I'm transferring source code from one platform to another... I will tell you the results later on. Thanks again, - edgar -- The author of this email does not necessarily endorse the following advertisements, which are the sole responsibility of the advertiser: |
From: edgar <edg...@we...> - 2010-04-20 12:48:04
|
Hello again CLISP list, Because this seems to become a longer conversation I would like to give some more information about me than only "edgar"... My full name is Edgar M. Franke, I live in Karlsruhe, Germany. (AFAIK Bruno Haible lived in Karlsruhe as a student in the early beginnings of the CLISP project.) I'm meanwhile 45 years of age as far as I can remember. I'm born in April 1965, for everything else use your computer. RFT is the German abbreviation for "Radio/Television broadcast technician", what you can imagine as "one step below the broadcast engineer". My main everyday business is building and repairing analog and digital recording equipment for sound and video studios. As an electronics hardware technician I am more familiar with Assembly opcode than with C compilers, so I'm one of the strange guys who uses Lisp for LDB, DBP, DISASSEMBLE and other funny things. When I started learnig Common Lisp approx. six years ago, CLISP was the first Common Lisp implementation I used. I also spent some time with SBCL (if I'm allowed to tell this here). I do not think of myself as a "Lisp expert", but as "fairly out of the worst trouble". I would like to use CLISP to introduce some friends of mine to the Common Lisp programming language, mainly because CLISP is much more tolerant against non-optimally written beginners code and because CLISP offers German error messages via a GNU 'gettext' interface. What leads to the next question(s): Looking at the 'de.po' file in CLISP CVS HEAD, I see that currently ony approx. 66% of the German strings are translated. Looking at the official CLISP translation page: http://translationproject.org/domain/clisp.html I see that the last official 'clisp-2.47-pre1.pot' file is from: "POT-Creation-Date: 2008-07-02 10:59:18-0400\n" "PO-Revision-Date: 2008-07-02 10:59:18-0400\n" "Last-Translator: Automatically generated <sds@loiso>\n" what is fairly obsolete. Because I'm aware that localization needs international synchronisation I would like to ask: * Is there any german translator except Bruno Haible? * Is there any chance to get an up-to-date 'official' po(t)-file? I would like to update the German translation during this summer because I have some people here who need German error messages. I would like to use the current CVS HEAD po-files instead of the obsolete 'official' files but also would like to re-contribute the translations back to the CLISP project. * What would be the best way how this could be managed? It's not too hurry, because next week the new Ubuntu will come out and then it will take another two or three weeks until all systems here are updated, I think it will be end of May until I can start the translations. Thanks so far, - edgar |
From: Sam S. <sd...@gn...> - 2010-04-20 17:41:22
|
edgar wrote: > > Because this seems to become a longer conversation I would like to > give some more information about me than only "edgar"... Welcome! > I also spent some time > with SBCL (if I'm allowed to tell this here). We are not jealous. Actually, one of the "official" application of clisp is bootstrapping sbcl :-) > Looking at the 'de.po' file in CLISP CVS HEAD, I see that currently > ony approx. 66% of the German strings are translated. > > Looking at the official CLISP translation page: > > http://translationproject.org/domain/clisp.html > > I see that the last official 'clisp-2.47-pre1.pot' file is from: > > "POT-Creation-Date: 2008-07-02 10:59:18-0400\n" > "PO-Revision-Date: 2008-07-02 10:59:18-0400\n" > "Last-Translator: Automatically generated <sds@loiso>\n" > > what is fairly obsolete. > > Because I'm aware that localization needs international > synchronisation I would like to ask: > > * Is there any german translator except Bruno Haible? no. > * Is there any chance to get an up-to-date 'official' po(t)-file? http://translationproject.org/domain/clisp.html is the official place. I submit the current pot files there a couple of weeks before releases. > I would like to update the German translation during this summer > because I have some people here who need German error messages. Thanks! Please contact the TP and tell them that you volunteer to translate the clisp messages, and they will set you up. > I would like to use the current CVS HEAD po-files instead of the > obsolete 'official' files but also would like to re-contribute > the translations back to the CLISP project. I think the best way would be for me to submit the new pot files to the TP when you will be ready to update the translations. Sam. |