From: <no...@so...> - 2001-07-09 12:05:09
|
Bugs item #439688, was opened at 2001-07-09 05:05 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=112867&aid=439688&group_id=12867 Category: Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Finn Bock (bckfnn) Assigned to: Finn Bock (bckfnn) Summary: Syntax error for non-ascii characters Initial Comment: When entering a big O-Umlaut in a euro-centric windows command prompt this happens: 1) Java reads it as 0x99. 2) Jython (wrongly) puts it trough the default encoding which return unicode 0x2122. 3) This is passed to the javaCC parser which assumes it is only dealing with ascii and cuts away the top 8 bits. 4) The result is 0x22 (a double quote) which cause a syntax error. >>> "Ö" Traceback (innermost last): (no code object) at line 0 File "<console>", line 2 SyntaxError: Lexical error at line 2, column 0. Encountered: <EOF> after : "" >>> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=112867&aid=439688&group_id=12867 |
From: <no...@so...> - 2001-07-19 19:18:43
|
Bugs item #439688, was opened at 2001-07-09 05:05 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=112867&aid=439688&group_id=12867 Category: Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Finn Bock (bckfnn) Assigned to: Finn Bock (bckfnn) Summary: Syntax error for non-ascii characters Initial Comment: When entering a big O-Umlaut in a euro-centric windows command prompt this happens: 1) Java reads it as 0x99. 2) Jython (wrongly) puts it trough the default encoding which return unicode 0x2122. 3) This is passed to the javaCC parser which assumes it is only dealing with ascii and cuts away the top 8 bits. 4) The result is 0x22 (a double quote) which cause a syntax error. >>> "Ö" Traceback (innermost last): (no code object) at line 0 File "<console>", line 2 SyntaxError: Lexical error at line 2, column 0. Encountered: <EOF> after : "" >>> ---------------------------------------------------------------------- >Comment By: Finn Bock (bckfnn) Date: 2001-07-19 12:18 Message: Logged In: YES user_id=4201 Added as test302 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=112867&aid=439688&group_id=12867 |
From: <no...@so...> - 2001-07-21 09:24:04
|
Bugs item #439688, was opened at 2001-07-09 05:05 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=112867&aid=439688&group_id=12867 Category: Core >Group: Fixed in 2.1a3 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Finn Bock (bckfnn) Assigned to: Finn Bock (bckfnn) Summary: Syntax error for non-ascii characters Initial Comment: When entering a big O-Umlaut in a euro-centric windows command prompt this happens: 1) Java reads it as 0x99. 2) Jython (wrongly) puts it trough the default encoding which return unicode 0x2122. 3) This is passed to the javaCC parser which assumes it is only dealing with ascii and cuts away the top 8 bits. 4) The result is 0x22 (a double quote) which cause a syntax error. >>> "Ö" Traceback (innermost last): (no code object) at line 0 File "<console>", line 2 SyntaxError: Lexical error at line 2, column 0. Encountered: <EOF> after : "" >>> ---------------------------------------------------------------------- >Comment By: Finn Bock (bckfnn) Date: 2001-07-21 02:24 Message: Logged In: YES user_id=4201 Fixed in parser.java: 2.10; python.jjt: 2.15. The parser now operate on the full unicode input as created by a default Reader. This fixes the syntax error but the using a default reader is still wrong when reading from the windows console. See patch #442906 for a way of fixing the encoding part of this bug. ---------------------------------------------------------------------- Comment By: Finn Bock (bckfnn) Date: 2001-07-19 12:18 Message: Logged In: YES user_id=4201 Added as test302 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=112867&aid=439688&group_id=12867 |