jython-checkins — For Jython checkin messages

[Jython-checkins] SF.net SVN: jython:[6517] trunk/jython

[<pj...@us...>]

Revision: 6517
          http://jython.svn.sourceforge.net/jython/?rev=6517&view=rev
Author:   pjenvey
Date:     2009-07-07 03:41:42 +0000 (Tue, 07 Jul 2009)

Log Message:
-----------
convert unicode hashlib input to str via defaultencoding
fixes #1189

Modified Paths:
--------------
    trunk/jython/NEWS
    trunk/jython/src/org/python/modules/_hashlib.java

Added Paths:
-----------
    trunk/jython/Lib/test/test_hashlib_jy.py

Added: trunk/jython/Lib/test/test_hashlib_jy.py
===================================================================
--- trunk/jython/Lib/test/test_hashlib_jy.py	                        (rev 0)
+++ trunk/jython/Lib/test/test_hashlib_jy.py	2009-07-07 03:41:42 UTC (rev 6517)
@@ -0,0 +1,23 @@
+# encoding: utf-8
+"""Misc hashlib tests
+
+Made for Jython.
+"""
+import hashlib
+import unittest
+from test import test_support
+
+class HashlibTestCase(unittest.TestCase):
+
+    def test_unicode(self):
+        self.assertEqual(hashlib.md5(u'foo').hexdigest(),
+                         'acbd18db4cc2f85cedef654fccc4a4d8')
+        self.assertRaises(UnicodeEncodeError, hashlib.md5, u'Gráin amháiñ')
+
+
+def test_main():
+    test_support.run_unittest(HashlibTestCase)
+
+
+if __name__ == '__main__':
+    test_main()

Modified: trunk/jython/NEWS
===================================================================
--- trunk/jython/NEWS	2009-07-07 02:24:24 UTC (rev 6516)
+++ trunk/jython/NEWS	2009-07-07 03:41:42 UTC (rev 6517)
@@ -9,6 +9,7 @@
     - [ 1365 ] continuation lines fail in interactive interpreter
     - [ 1377 ] Event names shadowed by a field name on Java types leads to a NPE
     - [ 1381 ] Redundant declarations of interface implementation hides overriden methods
+    - [ 1189 ] MD5 hash is incorrectly calculated when string contains non-latin chars and using python md5 lib
 
 Jython 2.5.0
   The same as rc4.

Modified: trunk/jython/src/org/python/modules/_hashlib.java
===================================================================
--- trunk/jython/src/org/python/modules/_hashlib.java	2009-07-07 02:24:24 UTC (rev 6516)
+++ trunk/jython/src/org/python/modules/_hashlib.java	2009-07-07 03:41:42 UTC (rev 6517)
@@ -7,10 +7,11 @@
 import java.util.Map;
 
 import org.python.core.ClassDictInit;
+import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
 import org.python.core.PyType;
-import org.python.core.Py;
+import org.python.core.PyUnicode;
 import org.python.core.util.StringUtil;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -105,7 +106,7 @@
 
         public static final PyType TYPE = PyType.fromClass(Hash.class);
 
-        /** The hash algorithm name */
+        /** The hash algorithm name. */
         @ExposedGet
         public String name;
 
@@ -121,14 +122,6 @@
                 put("sha-512", 128);
             }};
 
-        private static final MessageDigest getDigest(String name) {
-            try {
-                return MessageDigest.getInstance(name);
-            } catch (NoSuchAlgorithmException nsae) {
-                throw Py.ValueError("unsupported hash type");
-            }
-        }
-
         public Hash(String name) {
             this(name, getDigest(name));
         }
@@ -139,6 +132,14 @@
             this.digest = digest;
         }
 
+        private static final MessageDigest getDigest(String name) {
+            try {
+                return MessageDigest.getInstance(name);
+            } catch (NoSuchAlgorithmException nsae) {
+                throw Py.ValueError("unsupported hash type");
+            }
+        }
+
         /**
          * Clone the underlying MessageDigest.
          *
@@ -171,6 +172,9 @@
                 throw Py.TypeError("update() argument 1 must be string or read-only buffer, not "
                                    + obj.getType().fastGetName());
             }
+            if (obj instanceof PyUnicode) {
+                obj = obj.__str__();
+            }
             byte[] bytes = ((PyString)obj).toBytes();
             digest.update(bytes);
         }
@@ -232,6 +236,7 @@
             return Py.newInteger(size);
         }
 
+        @Override
         public String toString() {
             return String.format("<%s HASH object @ %s>", name, Py.idstr(this));
         }


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

Re: [Jython-checkins] SF.net SVN: jython:[6506] trunk/jython/Lib

[Philip J. <pj...@un...>]

On Jul 3, 2009, at 8:45 AM, am...@us... wrote:

> Revision: 6506
>          http://jython.svn.sourceforge.net/jython/?rev=6506&view=rev
> Author:   amak
> Date:     2009-07-03 15:45:08 +0000 (Fri, 03 Jul 2009)
>
> Log Message:
> -----------
> 1. Added support for the AI_PASSIVE flag
> 2. Added support for the AI_CANONNAME flag
> 3. Added unit tests for same
> 4. Split out getaddrinfo unit tests into their own test case class
>
> Modified Paths:
> --------------
>    trunk/jython/Lib/socket.py
>    trunk/jython/Lib/test/test_socket.py

FYI this broke all the buildbots:

http://bob.underboss.org:8080/job/jython/922/
http://freya.cs.uiuc.edu:8009/waterfall

--
Philip Jenvey

[Jython-checkins] SF.net SVN: jython:[6516] trunk/jython/grammar/Base.g

[<fwi...@us...>]

Revision: 6516
          http://jython.svn.sourceforge.net/jython/?rev=6516&view=rev
Author:   fwierzbicki
Date:     2009-07-07 02:24:24 +0000 (Tue, 07 Jul 2009)

Log Message:
-----------
Start of a Base grammar that is the common code from Python.g and
PythonPartial.g.

Added Paths:
-----------
    trunk/jython/grammar/Base.g

Added: trunk/jython/grammar/Base.g
===================================================================
--- trunk/jython/grammar/Base.g	                        (rev 0)
+++ trunk/jython/grammar/Base.g	2009-07-07 02:24:24 UTC (rev 6516)
@@ -0,0 +1,1192 @@
+/*
+ [The 'BSD licence']
+ Copyright (c) 2004 Terence Parr and Loring Craymer
+ All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+ 1. Redistributions of source code must retain the above copyright
+    notice, this list of conditions and the following disclaimer.
+ 2. Redistributions in binary form must reproduce the above copyright
+    notice, this list of conditions and the following disclaimer in the
+    documentation and/or other materials provided with the distribution.
+ 3. The name of the author may not be used to endorse or promote products
+    derived from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
+ IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+ IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
+ INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+ THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+*/
+
+/** Python 2.3.3 Grammar
+ *
+ *  Terence Parr and Loring Craymer
+ *  February 2004
+ *
+ *  Converted to ANTLR v3 November 2005 by Terence Parr.
+ *
+ *  This grammar was derived automatically from the Python 2.3.3
+ *  parser grammar to get a syntactically correct ANTLR grammar
+ *  for Python.  Then Terence hand tweaked it to be semantically
+ *  correct; i.e., removed lookahead issues etc...  It is LL(1)
+ *  except for the (sometimes optional) trailing commas and semi-colons.
+ *  It needs two symbols of lookahead in this case.
+ *
+ *  Starting with Loring's preliminary lexer for Python, I modified it
+ *  to do my version of the whole nasty INDENT/DEDENT issue just so I
+ *  could understand the problem better.  This grammar requires
+ *  PythonTokenStream.java to work.  Also I used some rules from the
+ *  semi-formal grammar on the web for Python (automatically
+ *  translated to ANTLR format by an ANTLR grammar, naturally <grin>).
+ *  The lexical rules for python are particularly nasty and it took me
+ *  a long time to get it 'right'; i.e., think about it in the proper
+ *  way.  Resist changing the lexer unless you've used ANTLR a lot. ;)
+ *
+ *  I (Terence) tested this by running it on the jython-2.1/Lib
+ *  directory of 40k lines of Python.
+ *
+ *  REQUIRES ANTLR v3
+ *
+ *  Updated to Python 2.5 by Frank Wierzbicki.
+ *
+ */
+
+grammar Base;
+
+@header {
+package org.python.antlr;
+}
+
+@members {
+
+    protected Object recoverFromMismatchedToken(IntStream input, int ttype, BitSet follow)
+        throws RecognitionException {
+
+        Object o = errorHandler.recoverFromMismatchedToken(this, input, ttype, follow);
+        if (o != null) {
+            return o;
+        }
+        return super.recoverFromMismatchedToken(input, ttype, follow);
+    }
+
+}
+
+@rulecatch {
+catch (RecognitionException re) {
+    errorHandler.reportError(this, re);
+    errorHandler.recover(this, input,re);
+    retval.tree = (PythonTree)adaptor.errorNode(input, retval.start, input.LT(-1), re);
+}
+}
+
+@lexer::header {
+package org.python.antlr;
+}
+
+@lexer::members {
+/** Handles context-sensitive lexing of implicit line joining such as
+ *  the case where newline is ignored in cases like this:
+ *  a = [3,
+ *       4]
+ */
+
+//For use in partial parsing.
+public boolean eofWhileNested = false;
+public boolean partial = false;
+
+int implicitLineJoiningLevel = 0;
+int startPos=-1;
+
+//If you want to use another error recovery mechanism change this
+//and the same one in the parser.
+private ErrorHandler errorHandler;
+
+    public void setErrorHandler(ErrorHandler eh) {
+        this.errorHandler = eh;
+    }
+
+    /**
+     *  Taken directly from antlr's Lexer.java -- needs to be re-integrated every time
+     *  we upgrade from Antlr (need to consider a Lexer subclass, though the issue would
+     *  remain).
+     */
+    public Token nextToken() {
+        while (true) {
+            state.token = null;
+            state.channel = Token.DEFAULT_CHANNEL;
+            state.tokenStartCharIndex = input.index();
+            state.tokenStartCharPositionInLine = input.getCharPositionInLine();
+            state.tokenStartLine = input.getLine();
+            state.text = null;
+            if ( input.LA(1)==CharStream.EOF ) {
+                if (implicitLineJoiningLevel > 0) {
+                    eofWhileNested = true;
+                }
+                return Token.EOF_TOKEN;
+            }
+            try {
+                mTokens();
+                if ( state.token==null ) {
+                    emit();
+                }
+                else if ( state.token==Token.SKIP_TOKEN ) {
+                    continue;
+                }
+                return state.token;
+            } catch (NoViableAltException nva) {
+                errorHandler.reportError(this, nva);
+                errorHandler.recover(this, nva); // throw out current char and try again
+            } catch (FailedPredicateException fp) {
+                //XXX: added this for failed STRINGPART -- the FailedPredicateException
+                //     hides a NoViableAltException.  This should be the only
+                //     FailedPredicateException that gets thrown by the lexer.
+                errorHandler.reportError(this, fp);
+                errorHandler.recover(this, fp); // throw out current char and try again
+            } catch (RecognitionException re) {
+                errorHandler.reportError(this, re);
+                // match() routine has already called recover()
+            }
+        }
+    }
+}
+
+
+//single_input: NEWLINE | simple_stmt | compound_stmt NEWLINE
+single_input
+    : NEWLINE* EOF
+    | simple_stmt NEWLINE* EOF
+    | compound_stmt NEWLINE+ EOF
+    ;
+
+//file_input: (NEWLINE | stmt)* ENDMARKER
+file_input
+    : (NEWLINE
+      | stmt
+      )* EOF
+    ;
+
+//eval_input: testlist NEWLINE* ENDMARKER
+eval_input
+    : LEADING_WS? (NEWLINE)* testlist (NEWLINE)* EOF
+    ;
+
+//not in CPython's Grammar file
+dotted_attr
+    : NAME
+      ( (DOT NAME)+
+      |
+      )
+    ;
+
+//attr is here for Java  compatibility.  A Java foo.getIf() can be called from Jython as foo.if
+//     so we need to support any keyword as an attribute.
+
+attr
+    : NAME
+    | AND
+    | AS
+    | ASSERT
+    | BREAK
+    | CLASS
+    | CONTINUE
+    | DEF
+    | DELETE
+    | ELIF
+    | EXCEPT
+    | EXEC
+    | FINALLY
+    | FROM
+    | FOR
+    | GLOBAL
+    | IF
+    | IMPORT
+    | IN
+    | IS
+    | LAMBDA
+    | NOT
+    | OR
+    | ORELSE
+    | PASS
+    | PRINT
+    | RAISE
+    | RETURN
+    | TRY
+    | WHILE
+    | WITH
+    | YIELD
+    ;
+
+//decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE
+decorator
+    : AT dotted_attr
+    ( LPAREN
+      ( arglist
+      |
+      )
+      RPAREN
+    |
+    ) NEWLINE
+    ;
+
+//decorators: decorator+
+decorators
+    : decorator+
+    ;
+
+//funcdef: [decorators] 'def' NAME parameters ':' suite
+funcdef
+    : decorators? DEF NAME parameters COLON suite
+    ;
+
+//parameters: '(' [varargslist] ')'
+parameters
+    : LPAREN
+      (varargslist
+      |
+      )
+      RPAREN
+    ;
+
+//not in CPython's Grammar file
+defparameter
+    : fpdef (ASSIGN test)?
+    ;
+
+//varargslist: ((fpdef ['=' test] ',')*
+//              ('*' NAME [',' '**' NAME] | '**' NAME) |
+//              fpdef ['=' test] (',' fpdef ['=' test])* [','])
+varargslist
+    : defparameter (options {greedy=true;}:COMMA defparameter)*
+      (COMMA
+          (STAR NAME (COMMA DOUBLESTAR NAME)?
+          | DOUBLESTAR NAME
+          )?
+      )?
+    | STAR NAME (COMMA DOUBLESTAR NAME)?
+    | DOUBLESTAR NAME
+    ;
+
+//fpdef: NAME | '(' fplist ')'
+fpdef
+    : NAME
+    | (LPAREN fpdef COMMA) => LPAREN fplist RPAREN
+    | LPAREN fplist RPAREN
+    ;
+
+//fplist: fpdef (',' fpdef)* [',']
+fplist
+    : fpdef
+      (options {greedy=true;}:COMMA fpdef)* (COMMA)?
+    ;
+
+//stmt: simple_stmt | compound_stmt
+stmt
+    : simple_stmt
+    | compound_stmt
+    ;
+
+//simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE
+simple_stmt
+    : small_stmt (options {greedy=true;}:SEMI small_stmt)* (SEMI)? NEWLINE
+    ;
+
+//small_stmt: (expr_stmt | print_stmt  | del_stmt | pass_stmt | flow_stmt |
+//             import_stmt | global_stmt | exec_stmt | assert_stmt)
+small_stmt : expr_stmt
+           | print_stmt
+           | del_stmt
+           | pass_stmt
+           | flow_stmt
+           | import_stmt
+           | global_stmt
+           | exec_stmt
+           | assert_stmt
+           ;
+
+//expr_stmt: testlist (augassign (yield_expr|testlist) |
+//                     ('=' (yield_expr|testlist))*)
+expr_stmt
+    : ((testlist augassign) => testlist
+        ( (augassign yield_expr
+          )
+        | (augassign testlist
+          )
+        )
+    | (testlist ASSIGN) => testlist
+        (
+        | ((ASSIGN testlist)+
+          )
+        | ((ASSIGN yield_expr)+
+          )
+        )
+    | testlist
+    )
+    ;
+
+//augassign: ('+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' |
+//            '<<=' | '>>=' | '**=' | '//=')
+augassign
+    : PLUSEQUAL
+    | MINUSEQUAL
+    | STAREQUAL
+    | SLASHEQUAL
+    | PERCENTEQUAL
+    | AMPEREQUAL
+    | VBAREQUAL
+    | CIRCUMFLEXEQUAL
+    | LEFTSHIFTEQUAL
+    | RIGHTSHIFTEQUAL
+    | DOUBLESTAREQUAL
+    | DOUBLESLASHEQUAL
+    ;
+
+//print_stmt: 'print' ( [ test (',' test)* [','] ] |
+//                      '>>' test [ (',' test)+ [','] ] )
+print_stmt
+    : PRINT
+      (printlist
+      | RIGHTSHIFT printlist2
+      |
+      )
+      ;
+
+//not in CPython's Grammar file
+printlist
+    : (test COMMA) =>
+       test (options {k=2;}: COMMA test)*
+         (COMMA)?
+    | test
+    ;
+
+//XXX: would be nice if printlist and printlist2 could be merged.
+//not in CPython's Grammar file
+printlist2
+    : (test COMMA test]) =>
+       test (options {k=2;}: COMMA test)*
+         (COMMA)?
+    | test
+    ;
+
+//del_stmt: 'del' exprlist
+del_stmt
+    : DELETE del_list
+    ;
+
+//pass_stmt: 'pass'
+pass_stmt
+    : PASS
+    ;
+
+//flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt
+flow_stmt
+    : break_stmt
+    | continue_stmt
+    | return_stmt
+    | raise_stmt
+    | yield_stmt
+    ;
+
+//break_stmt: 'break'
+break_stmt
+    : BREAK
+    ;
+
+//continue_stmt: 'continue'
+continue_stmt
+    : CONTINUE
+    ;
+
+//return_stmt: 'return' [testlist]
+return_stmt
+    : RETURN
+      (testlist
+      |
+      )
+    ;
+
+//yield_stmt: yield_expr
+yield_stmt
+    : yield_expr
+    ;
+
+//raise_stmt: 'raise' [test [',' test [',' test]]]
+raise_stmt
+    : RAISE (test (COMMA test
+        (COMMA test)?)?)?
+    ;
+
+//import_stmt: import_name | import_from
+import_stmt
+    : import_name
+    | import_from
+    ;
+
+//import_name: 'import' dotted_as_names
+import_name
+    : IMPORT dotted_as_names
+    ;
+
+//import_from: ('from' ('.'* dotted_name | '.'+)
+//              'import' ('*' | '(' import_as_names ')' | import_as_names))
+import_from
+    : FROM (DOT* dotted_name | DOT+) IMPORT
+        (STAR
+        | import_as_names
+        | LPAREN import_as_names COMMA? RPAREN
+        )
+    ;
+
+//import_as_names: import_as_name (',' import_as_name)* [',']
+import_as_names
+    : import_as_name (COMMA import_as_name)*
+    ;
+
+//import_as_name: NAME [('as' | NAME) NAME]
+import_as_name
+    : NAME (AS NAME)?
+    ;
+
+//XXX: when does CPython Grammar match "dotted_name NAME NAME"?
+//dotted_as_name: dotted_name [('as' | NAME) NAME]
+dotted_as_name
+    : dotted_name (AS NAME)?
+    ;
+
+//dotted_as_names: dotted_as_name (',' dotted_as_name)*
+dotted_as_names
+    : dotted_as_name (COMMA dotted_as_name)*
+    ;
+
+//dotted_name: NAME ('.' NAME)*
+dotted_name
+    : NAME (DOT attr)*
+    ;
+
+//global_stmt: 'global' NAME (',' NAME)*
+global_stmt
+    : GLOBAL NAME (COMMA NAME)*
+    ;
+
+//exec_stmt: 'exec' expr ['in' test [',' test]]
+exec_stmt
+    : EXEC expr (IN test (COMMA test)?)?
+    ;
+
+//assert_stmt: 'assert' test [',' test]
+assert_stmt
+    : ASSERT test (COMMA test)?
+    ;
+
+//compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | funcdef | classdef
+compound_stmt
+    : if_stmt
+    | while_stmt
+    | for_stmt
+    | try_stmt
+    | with_stmt
+    | (decorators? DEF) => funcdef
+    | classdef
+    ;
+
+//if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
+if_stmt
+    : IF test COLON suite elif_clause?
+    ;
+
+//not in CPython's Grammar file
+elif_clause
+    : else_clause
+    | ELIF test COLON suite
+        (elif_clause
+        |
+        )
+    ;
+
+//not in CPython's Grammar file
+else_clause
+    : ORELSE COLON suite
+    ;
+
+//while_stmt: 'while' test ':' suite ['else' ':' suite]
+while_stmt
+    : WHILE test COLON suite (ORELSE COLON suite)?
+    ;
+
+//for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]
+for_stmt
+    : FOR exprlist IN testlist COLON suite
+        (ORELSE COLON suite)?
+    ;
+
+//try_stmt: ('try' ':' suite
+//           ((except_clause ':' suite)+
+//           ['else' ':' suite]
+//           ['finally' ':' suite] |
+//           'finally' ':' suite))
+try_stmt
+    : TRY COLON suite
+      ( except_clause+ (ORELSE COLON suite)? (FINALLY COLON suite)?
+      | FINALLY COLON suite
+      )
+      ;
+
+//with_stmt: 'with' test [ with_var ] ':' suite
+with_stmt
+    : WITH test (with_var)? COLON suite
+    ;
+
+//with_var: ('as' | NAME) expr
+with_var
+    : (AS | NAME) expr
+    ;
+
+//except_clause: 'except' [test [',' test]]
+except_clause
+    : EXCEPT (test (COMMA test)?)? COLON suite
+    ;
+
+//suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT
+suite
+    : simple_stmt
+    | NEWLINE INDENT
+      (stmt
+      )+ DEDENT
+    ;
+
+//test: or_test ['if' or_test 'else' test] | lambdef
+test
+    :or_test
+      ( (IF or_test ORELSE) => IF or_test ORELSE test
+      |
+      )
+    | lambdef
+    ;
+
+//or_test: and_test ('or' and_test)*
+or_test
+    : and_test
+        ( (OR and_test
+          )+
+        |
+        )
+    ;
+
+//and_test: not_test ('and' not_test)*
+and_test
+    : not_test
+        ( (AND not_test
+          )+
+        |
+        )
+    ;
+
+//not_test: 'not' not_test | comparison
+not_test
+    : NOT not_test
+    | comparison
+    ;
+
+//comparison: expr (comp_op expr)*
+comparison
+    : expr
+       ( ( comp_op expr
+         )+
+       |
+       )
+    ;
+
+//comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not'
+comp_op
+    : LESS
+    | GREATER
+    | EQUAL
+    | GREATEREQUAL
+    | LESSEQUAL
+    | ALT_NOTEQUAL
+    | NOTEQUAL
+    | IN
+    | NOT IN
+    | IS
+    | IS NOT
+    ;
+
+//expr: xor_expr ('|' xor_expr)*
+expr
+    : xor_expr
+        ( (VBAR xor_expr
+          )+
+        |
+        )
+    ;
+
+//xor_expr: and_expr ('^' and_expr)*
+xor_expr
+    : and_expr
+        ( (CIRCUMFLEX and_expr
+          )+
+        |
+        )
+    ;
+
+//and_expr: shift_expr ('&' shift_expr)*
+and_expr
+    : shift_expr
+        ( (AMPER shift_expr
+          )+
+        |
+        )
+    ;
+
+//shift_expr: arith_expr (('<<'|'>>') arith_expr)*
+shift_expr
+    : arith_expr
+        ( ( shift_op arith_expr
+          )+
+        |
+        )
+    ;
+
+shift_op
+    : LEFTSHIFT
+    | RIGHTSHIFT
+    ;
+
+//arith_expr: term (('+'|'-') term)*
+arith_expr
+    : term
+        ( (arith_op term
+          )+
+        |
+        )
+    ;
+
+arith_op
+    : PLUS
+    | MINUS
+    ;
+
+//term: factor (('*'|'/'|'%'|'//') factor)*
+term
+    : factor
+        ( (term_op factor
+          )+
+        |
+        )
+    ;
+
+term_op
+    :STAR
+    |SLASH
+    |PERCENT
+    |DOUBLESLASH
+    ;
+
+//factor: ('+'|'-'|'~') factor | power
+factor
+    : PLUS factor
+    | MINUS factor
+    | TILDE factor
+    | power
+    ;
+
+//power: atom trailer* ['**' factor]
+power
+    : atom (trailer)* (options {greedy=true;}:DOUBLESTAR factor)?
+    ;
+
+//atom: ('(' [yield_expr|testlist_gexp] ')' |
+//       '[' [listmaker] ']' |
+//       '{' [dictmaker] '}' |
+//       '`' testlist1 '`' |
+//       NAME | NUMBER | STRING+)
+atom
+    : LPAREN
+      ( yield_expr
+      | testlist_gexp
+      |
+      )
+      RPAREN
+    | LBRACK
+      (listmaker
+      |
+      )
+      RBRACK
+    | LCURLY
+       (dictmaker
+       |
+       )
+       RCURLY
+     | BACKQUOTE testlist BACKQUOTE
+     | NAME
+     | INT
+     | LONGINT
+     | FLOAT
+     | COMPLEX
+     | (STRING)+
+     ;
+
+//listmaker: test ( list_for | (',' test)* [','] )
+listmaker
+    : test
+        (list_for
+        | (options {greedy=true;}:COMMA test)*
+        ) (COMMA)?
+          ;
+
+//testlist_gexp: test ( gen_for | (',' test)* [','] )
+testlist_gexp
+    : test
+        ( ((options {k=2;}: COMMA test)* (COMMA)?
+          )
+        | (gen_for
+          )
+        )
+    ;
+
+//lambdef: 'lambda' [varargslist] ':' test
+lambdef
+    : LAMBDA (varargslist)? COLON test
+    ;
+
+//trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME
+trailer
+    : LPAREN
+        (arglist
+        |
+        )
+      RPAREN
+    | LBRACK subscriptlist RBRACK
+    | DOT attr
+    ;
+
+//subscriptlist: subscript (',' subscript)* [',']
+subscriptlist
+    : subscript (options {greedy=true;}:COMMA subscript)* (COMMA)?
+    ;
+
+//subscript: '.' '.' '.' | test | [test] ':' [test] [sliceop]
+subscript
+    : DOT DOT DOT
+    | (test COLON)
+   => test (COLON (test)? (sliceop)?)?
+    | (COLON)
+   => COLON (test)? (sliceop)?
+    | test
+    ;
+
+//sliceop: ':' [test]
+sliceop
+    : COLON
+     (test
+     |
+     )
+    ;
+
+//exprlist: expr (',' expr)* [',']
+exprlist
+    : (expr COMMA) => expr (options {k=2;}: COMMA expr)* (COMMA)?
+    | expr
+    ;
+
+//not in CPython's Grammar file
+//Needed as an exprlist that does not produce tuples for del_stmt.
+del_list
+    : expr (options {k=2;}: COMMA expr)* (COMMA)?
+    ;
+
+//testlist: test (',' test)* [',']
+testlist
+    : (test COMMA)
+   => test (options {k=2;}: COMMA test)* (COMMA)?
+    | test
+    ;
+
+//dictmaker: test ':' test (',' test ':' test)* [',']
+dictmaker
+    : test COLON test
+        (options {k=2;}:COMMA test COLON test)*
+        (COMMA)?
+    ;
+
+//classdef: 'class' NAME ['(' [testlist] ')'] ':' suite
+classdef
+    : decorators? CLASS NAME (LPAREN testlist? RPAREN)? COLON suite
+    ;
+
+//arglist: (argument ',')* (argument [',']| '*' test [',' '**' test] | '**' test)
+arglist
+    : argument (COMMA argument)*
+          (COMMA
+              ( STAR test (COMMA DOUBLESTAR test)?
+              | DOUBLESTAR test
+              )?
+          )?
+    | STAR test (COMMA DOUBLESTAR test)?
+    | DOUBLESTAR test
+    ;
+
+//argument: test [gen_for] | test '=' test  # Really [keyword '='] test
+argument
+    : test
+        ((ASSIGN test)
+        | gen_for
+        |
+        )
+    ;
+
+//list_iter: list_for | list_if
+list_iter
+    : list_for
+    | list_if
+    ;
+
+//list_for: 'for' exprlist 'in' testlist_safe [list_iter]
+list_for
+    : FOR exprlist IN testlist (list_iter)?
+    ;
+
+//list_if: 'if' test [list_iter]
+list_if
+    : IF test (list_iter)?
+    ;
+
+//gen_iter: gen_for | gen_if
+gen_iter
+    : gen_for
+    | gen_if
+    ;
+
+//gen_for: 'for' exprlist 'in' or_test [gen_iter]
+gen_for
+    : FOR exprlist IN or_test gen_iter?
+    ;
+
+//gen_if: 'if' old_test [gen_iter]
+gen_if
+    : IF test gen_iter?
+    ;
+
+//yield_expr: 'yield' [testlist]
+yield_expr
+    : YIELD testlist?
+    ;
+
+AS        : 'as' ;
+ASSERT    : 'assert' ;
+BREAK     : 'break' ;
+CLASS     : 'class' ;
+CONTINUE  : 'continue' ;
+DEF       : 'def' ;
+DELETE    : 'del' ;
+ELIF      : 'elif' ;
+EXCEPT    : 'except' ;
+EXEC      : 'exec' ;
+FINALLY   : 'finally' ;
+FROM      : 'from' ;
+FOR       : 'for' ;
+GLOBAL    : 'global' ;
+IF        : 'if' ;
+IMPORT    : 'import' ;
+IN        : 'in' ;
+IS        : 'is' ;
+LAMBDA    : 'lambda' ;
+ORELSE    : 'else' ;
+PASS      : 'pass'  ;
+PRINT     : 'print' ;
+RAISE     : 'raise' ;
+RETURN    : 'return' ;
+TRY       : 'try' ;
+WHILE     : 'while' ;
+WITH      : 'with' ;
+YIELD     : 'yield' ;
+
+LPAREN    : '(' {implicitLineJoiningLevel++;} ;
+
+RPAREN    : ')' {implicitLineJoiningLevel--;} ;
+
+LBRACK    : '[' {implicitLineJoiningLevel++;} ;
+
+RBRACK    : ']' {implicitLineJoiningLevel--;} ;
+
+COLON     : ':' ;
+
+COMMA    : ',' ;
+
+SEMI    : ';' ;
+
+PLUS    : '+' ;
+
+MINUS    : '-' ;
+
+STAR    : '*' ;
+
+SLASH    : '/' ;
+
+VBAR    : '|' ;
+
+AMPER    : '&' ;
+
+LESS    : '<' ;
+
+GREATER    : '>' ;
+
+ASSIGN    : '=' ;
+
+PERCENT    : '%' ;
+
+BACKQUOTE    : '`' ;
+
+LCURLY    : '{' {implicitLineJoiningLevel++;} ;
+
+RCURLY    : '}' {implicitLineJoiningLevel--;} ;
+
+CIRCUMFLEX    : '^' ;
+
+TILDE    : '~' ;
+
+EQUAL    : '==' ;
+
+NOTEQUAL    : '!=' ;
+
+ALT_NOTEQUAL: '<>' ;
+
+LESSEQUAL    : '<=' ;
+
+LEFTSHIFT    : '<<' ;
+
+GREATEREQUAL    : '>=' ;
+
+RIGHTSHIFT    : '>>' ;
+
+PLUSEQUAL    : '+=' ;
+
+MINUSEQUAL    : '-=' ;
+
+DOUBLESTAR    : '**' ;
+
+STAREQUAL    : '*=' ;
+
+DOUBLESLASH    : '//' ;
+
+SLASHEQUAL    : '/=' ;
+
+VBAREQUAL    : '|=' ;
+
+PERCENTEQUAL    : '%=' ;
+
+AMPEREQUAL    : '&=' ;
+
+CIRCUMFLEXEQUAL    : '^=' ;
+
+LEFTSHIFTEQUAL    : '<<=' ;
+
+RIGHTSHIFTEQUAL    : '>>=' ;
+
+DOUBLESTAREQUAL    : '**=' ;
+
+DOUBLESLASHEQUAL    : '//=' ;
+
+DOT : '.' ;
+
+AT : '@' ;
+
+AND : 'and' ;
+
+OR : 'or' ;
+
+NOT : 'not' ;
+
+FLOAT
+    :   '.' DIGITS (Exponent)?
+    |   DIGITS '.' Exponent
+    |   DIGITS ('.' (DIGITS (Exponent)?)? | Exponent)
+    ;
+
+LONGINT
+    :   INT ('l'|'L')
+    ;
+
+fragment
+Exponent
+    :    ('e' | 'E') ( '+' | '-' )? DIGITS
+    ;
+
+INT :   // Hex
+        '0' ('x' | 'X') ( '0' .. '9' | 'a' .. 'f' | 'A' .. 'F' )+
+    |   // Octal
+        '0'  ( '0' .. '7' )*
+    |   '1'..'9' DIGITS*
+    ;
+
+COMPLEX
+    :   DIGITS+ ('j'|'J')
+    |   FLOAT ('j'|'J')
+    ;
+
+fragment
+DIGITS : ( '0' .. '9' )+ ;
+
+NAME:    ( 'a' .. 'z' | 'A' .. 'Z' | '_')
+        ( 'a' .. 'z' | 'A' .. 'Z' | '_' | '0' .. '9' )*
+    ;
+
+/** Match various string types.  Note that greedy=false implies '''
+ *  should make us exit loop not continue.
+ */
+STRING
+    :   ('r'|'u'|'ur'|'R'|'U'|'UR'|'uR'|'Ur')?
+        (   '\'\'\'' (options {greedy=false;}:TRIAPOS)* '\'\'\''
+        |   '"""' (options {greedy=false;}:TRIQUOTE)* '"""'
+        |   '"' (ESC|~('\\'|'\n'|'"'))* '"'
+        |   '\'' (ESC|~('\\'|'\n'|'\''))* '\''
+        ) {
+           if (state.tokenStartLine != input.getLine()) {
+               state.tokenStartLine = input.getLine();
+               state.tokenStartCharPositionInLine = -2;
+           }
+        }
+    ;
+
+/** the two '"'? cause a warning -- is there a way to avoid that? */
+fragment
+TRIQUOTE
+    : '"'? '"'? (ESC|~('\\'|'"'))+
+    ;
+
+/** the two '\''? cause a warning -- is there a way to avoid that? */
+fragment
+TRIAPOS
+    : '\''? '\''? (ESC|~('\\'|'\''))+
+    ;
+
+fragment
+ESC
+    :    '\\' .
+    ;
+
+/** Consume a newline and any whitespace at start of next line
+ *  unless the next line contains only white space, in that case
+ *  emit a newline.
+ */
+CONTINUED_LINE
+    :    '\\' ('\r')? '\n' (' '|'\t')*  { $channel=HIDDEN; }
+         ( COMMENT
+         | nl=NEWLINE
+           {
+               emit(new CommonToken(NEWLINE,nl.getText()));
+           }
+         |
+         ) {
+               if (input.LA(1) == -1) {
+                   throw new ParseException("unexpected character after line continuation character");
+               }
+           }
+    ;
+
+/** Treat a sequence of blank lines as a single blank line.  If
+ *  nested within a (..), {..}, or [..], then ignore newlines.
+ *  If the first newline starts in column one, they are to be ignored.
+ *
+ *  Frank Wierzbicki added: Also ignore FORMFEEDS (\u000C).
+ */
+NEWLINE
+@init {
+    int newlines = 0;
+}
+    :   (('\u000C')?('\r')? '\n' {newlines++; } )+ {
+         if ( startPos==0 || implicitLineJoiningLevel>0 )
+            $channel=HIDDEN;
+        }
+    ;
+
+WS  :    {startPos>0}?=> (' '|'\t'|'\u000C')+ {$channel=HIDDEN;}
+    ;
+
+/** Grab everything before a real symbol.  Then if newline, kill it
+ *  as this is a blank line.  If whitespace followed by comment, kill it
+ *  as it's a comment on a line by itself.
+ *
+ *  Ignore leading whitespace when nested in [..], (..), {..}.
+ */
+LEADING_WS
+@init {
+    int spaces = 0;
+    int newlines = 0;
+}
+    :   {startPos==0}?=>
+        (   {implicitLineJoiningLevel>0}? ( ' ' | '\t' )+ {$channel=HIDDEN;}
+        |    (     ' '  { spaces++; }
+             |    '\t' { spaces += 8; spaces -= (spaces \% 8); }
+             )+
+             ( ('\r')? '\n' {newlines++; }
+             )* {
+                   if (input.LA(1) != -1 || newlines == 0) {
+                       // make a string of n spaces where n is column number - 1
+                       char[] indentation = new char[spaces];
+                       for (int i=0; i<spaces; i++) {
+                           indentation[i] = ' ';
+                       }
+                       CommonToken c = new CommonToken(LEADING_WS,new String(indentation));
+                       c.setLine(input.getLine());
+                       c.setCharPositionInLine(input.getCharPositionInLine());
+                       c.setStartIndex(input.index() - 1);
+                       c.setStopIndex(input.index() - 1);
+                       emit(c);
+                       // kill trailing newline if present and then ignore
+                       if (newlines != 0) {
+                           if (state.token!=null) {
+                               state.token.setChannel(HIDDEN);
+                           } else {
+                               $channel=HIDDEN;
+                           }
+                       }
+                   } else {
+                       // make a string of n newlines
+                       char[] nls = new char[newlines];
+                       for (int i=0; i<newlines; i++) {
+                           nls[i] = '\n';
+                       }
+                       CommonToken c = new CommonToken(NEWLINE,new String(nls));
+                       c.setLine(input.getLine());
+                       c.setCharPositionInLine(input.getCharPositionInLine());
+                       c.setStartIndex(input.index() - 1);
+                       c.setStopIndex(input.index() - 1);
+                       emit(c);
+                   }
+                }
+        )
+    ;
+
+/** Comments not on line by themselves are turned into newlines.
+
+    b = a # end of line comment
+
+    or
+
+    a = [1, # weird
+         2]
+
+    This rule is invoked directly by nextToken when the comment is in
+    first column or when comment is on end of nonwhitespace line.
+
+    Only match \n here if we didn't start on left edge; let NEWLINE return that.
+    Kill if newlines if we live on a line by ourselves
+
+    Consume any leading whitespace if it starts on left edge.
+ */
+COMMENT
+@init {
+    $channel=HIDDEN;
+}
+    :    {startPos==0}?=> (' '|'\t')* '#' (~'\n')* '\n'+
+    |    '#' (~'\n')* // let NEWLINE handle \n unless char pos==0 for '#'
+    ;
+


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6515] trunk/jython/grammar/Python.g

[<fwi...@us...>]

Revision: 6515
          http://jython.svn.sourceforge.net/jython/?rev=6515&view=rev
Author:   fwierzbicki
Date:     2009-07-06 18:02:32 +0000 (Mon, 06 Jul 2009)

Log Message:
-----------
Whitespace cleanup.

Modified Paths:
--------------
    trunk/jython/grammar/Python.g

Modified: trunk/jython/grammar/Python.g
===================================================================
--- trunk/jython/grammar/Python.g	2009-07-06 16:31:40 UTC (rev 6514)
+++ trunk/jython/grammar/Python.g	2009-07-06 18:02:32 UTC (rev 6515)
@@ -274,15 +274,18 @@
 @after {
     $single_input.tree = mtype;
 }
-    : NEWLINE* EOF {
+    : NEWLINE* EOF
+      {
         mtype = new Interactive($single_input.start, new ArrayList<stmt>());
-    }
-    | simple_stmt NEWLINE* EOF {
+      }
+    | simple_stmt NEWLINE* EOF
+      {
         mtype = new Interactive($single_input.start, actions.castStmts($simple_stmt.stypes));
-    }
-    | compound_stmt NEWLINE+ EOF {
+      }
+    | compound_stmt NEWLINE+ EOF
+      {
         mtype = new Interactive($single_input.start, actions.castStmts($compound_stmt.tree));
-    }
+      }
     ;
     //XXX: this block is duplicated in three places, how to extract?
     catch [RecognitionException re] {
@@ -312,7 +315,8 @@
     $file_input.tree = mtype;
 }
     : (NEWLINE
-      | stmt {
+      | stmt
+      {
           if ($stmt.stypes != null)
                  {stypes.addAll($stmt.stypes);}
       }
@@ -337,9 +341,10 @@
 @after {
     $eval_input.tree = mtype;
 }
-    : LEADING_WS? (NEWLINE)* testlist[expr_contextType.Load] (NEWLINE)* EOF {
+    : LEADING_WS? (NEWLINE)* testlist[expr_contextType.Load] (NEWLINE)* EOF
+      {
         mtype = new Expression($eval_input.start, actions.castExpr($testlist.tree));
-    }
+      }
     ;
     //XXX: this block is duplicated in three places, how to extract?
     catch [RecognitionException re] {
@@ -351,10 +356,17 @@
 
 
 //not in CPython's Grammar file
-dotted_attr returns [expr etype]
+dotted_attr
+    returns [expr etype]
     : n1=NAME
-      ( (DOT n2+=NAME)+ { $etype = actions.makeDottedAttr($n1, $n2); }
-      | { $etype = new Name($n1, $n1.text, expr_contextType.Load); }
+      ( (DOT n2+=NAME)+
+        {
+            $etype = actions.makeDottedAttr($n1, $n2);
+        }
+      |
+        {
+            $etype = new Name($n1, $n1.text, expr_contextType.Load);
+        }
       )
     ;
 
@@ -397,7 +409,8 @@
     ;
 
 //decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE
-decorator returns [expr etype]
+decorator
+    returns [expr etype]
 @after {
    $decorator.tree = $etype;
 }
@@ -408,19 +421,22 @@
             $etype = actions.makeCall($LPAREN, $dotted_attr.etype, $arglist.args, $arglist.keywords,
                      $arglist.starargs, $arglist.kwargs);
         }
-      | {
+      |
+        {
             $etype = actions.makeCall($LPAREN, $dotted_attr.etype);
         }
       )
       RPAREN
-    | {
+    |
+      {
           $etype = $dotted_attr.etype;
       }
     ) NEWLINE
     ;
 
 //decorators: decorator+
-decorators returns [List etypes]
+decorators
+    returns [List etypes]
     : d+=decorator+
       {
           $etypes = $d;
@@ -442,19 +458,24 @@
     ;
 
 //parameters: '(' [varargslist] ')'
-parameters returns [arguments args]
+parameters
+    returns [arguments args]
     : LPAREN
       (varargslist
-          {$args = $varargslist.args;
-          }
-      | { $args = new arguments($parameters.start, new ArrayList<expr>(), null, null, new ArrayList<expr>());
+        {
+              $args = $varargslist.args;
         }
+      |
+        {
+            $args = new arguments($parameters.start, new ArrayList<expr>(), null, null, new ArrayList<expr>());
+        }
       )
       RPAREN
     ;
 
 //not in CPython's Grammar file
-defparameter[List defaults] returns [expr etype]
+defparameter
+    [List defaults] returns [expr etype]
 @after {
    $defparameter.tree = $etype;
 }
@@ -472,7 +493,8 @@
 //varargslist: ((fpdef ['=' test] ',')*
 //              ('*' NAME [',' '**' NAME] | '**' NAME) |
 //              fpdef ['=' test] (',' fpdef ['=' test])* [','])
-varargslist returns [arguments args]
+varargslist
+    returns [arguments args]
 @init {
     List defaults = new ArrayList();
 }
@@ -509,7 +531,8 @@
     ;
 
 //fplist: fpdef (',' fpdef)* [',']
-fplist returns [List etypes]
+fplist
+    returns [List etypes]
     : f+=fpdef[expr_contextType.Store]
       (options {greedy=true;}:COMMA f+=fpdef[expr_contextType.Store])* (COMMA)?
       {
@@ -518,7 +541,8 @@
     ;
 
 //stmt: simple_stmt | compound_stmt
-stmt returns [List stypes]
+stmt
+    returns [List stypes]
     : simple_stmt
       {
           $stypes = $simple_stmt.stypes;
@@ -531,7 +555,8 @@
     ;
 
 //simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE
-simple_stmt returns [List stypes]
+simple_stmt
+    returns [List stypes]
     : s+=small_stmt (options {greedy=true;}:SEMI s+=small_stmt)* (SEMI)? NEWLINE
       {
           $stypes = $s;
@@ -564,16 +589,16 @@
 }
     : ((testlist[null] augassign) => lhs=testlist[expr_contextType.AugStore]
         ( (aay=augassign y1=yield_expr
-            {
-                actions.checkAssign(actions.castExpr($lhs.tree));
-                stype = new AugAssign($lhs.tree, actions.castExpr($lhs.tree), $aay.op, actions.castExpr($y1.tree));
-            }
+           {
+               actions.checkAssign(actions.castExpr($lhs.tree));
+               stype = new AugAssign($lhs.tree, actions.castExpr($lhs.tree), $aay.op, actions.castExpr($y1.tree));
+           }
           )
         | (aat=augassign rhs=testlist[expr_contextType.Load]
-            {
-                actions.checkAssign(actions.castExpr($lhs.tree));
-                stype = new AugAssign($lhs.tree, actions.castExpr($lhs.tree), $aat.op, actions.castExpr($rhs.tree));
-            }
+           {
+               actions.checkAssign(actions.castExpr($lhs.tree));
+               stype = new AugAssign($lhs.tree, actions.castExpr($lhs.tree), $aat.op, actions.castExpr($rhs.tree));
+           }
           )
         )
     | (testlist[null] ASSIGN) => lhs=testlist[expr_contextType.Store]
@@ -596,19 +621,32 @@
 
 //augassign: ('+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' |
 //            '<<=' | '>>=' | '**=' | '//=')
-augassign returns [operatorType op]
-    : PLUSEQUAL {$op = operatorType.Add;}
-    | MINUSEQUAL {$op = operatorType.Sub;}
-    | STAREQUAL {$op = operatorType.Mult;}
-    | SLASHEQUAL {$op = operatorType.Div;}
-    | PERCENTEQUAL {$op = operatorType.Mod;}
-    | AMPEREQUAL {$op = operatorType.BitAnd;}
-    | VBAREQUAL {$op = operatorType.BitOr;}
-    | CIRCUMFLEXEQUAL {$op = operatorType.BitXor;}
-    | LEFTSHIFTEQUAL {$op = operatorType.LShift;}
-    | RIGHTSHIFTEQUAL {$op = operatorType.RShift;}
-    | DOUBLESTAREQUAL {$op = operatorType.Pow;}
-    | DOUBLESLASHEQUAL {$op = operatorType.FloorDiv;}
+augassign
+    returns [operatorType op]
+    : PLUSEQUAL
+        {$op = operatorType.Add;}
+    | MINUSEQUAL
+        {$op = operatorType.Sub;}
+    | STAREQUAL
+        {$op = operatorType.Mult;}
+    | SLASHEQUAL
+        {$op = operatorType.Div;}
+    | PERCENTEQUAL
+        {$op = operatorType.Mod;}
+    | AMPEREQUAL
+        {$op = operatorType.BitAnd;}
+    | VBAREQUAL
+        {$op = operatorType.BitOr;}
+    | CIRCUMFLEXEQUAL
+        {$op = operatorType.BitXor;}
+    | LEFTSHIFTEQUAL
+        {$op = operatorType.LShift;}
+    | RIGHTSHIFTEQUAL
+        {$op = operatorType.RShift;}
+    | DOUBLESTAREQUAL
+        {$op = operatorType.Pow;}
+    | DOUBLESLASHEQUAL
+        {$op = operatorType.FloorDiv;}
     ;
 
 //print_stmt: 'print' ( [ test (',' test)* [','] ] |
@@ -625,10 +663,10 @@
       ;
 
 //not in CPython's Grammar file
-printlist returns [boolean newline, List elts]
+printlist
+    returns [boolean newline, List elts]
     : (test[null] COMMA) =>
-       t+=test[expr_contextType.Load] (options {k=2;}: COMMA t+=test[expr_contextType.Load])*
-         (trailcomma=COMMA)?
+       t+=test[expr_contextType.Load] (options {k=2;}: COMMA t+=test[expr_contextType.Load])* (trailcomma=COMMA)?
        {
            $elts=$t;
            if ($trailcomma == null) {
@@ -646,10 +684,10 @@
 
 //XXX: would be nice if printlist and printlist2 could be merged.
 //not in CPython's Grammar file
-printlist2 returns [boolean newline, List elts]
+printlist2
+    returns [boolean newline, List elts]
     : (test[null] COMMA test[null]) =>
-       t+=test[expr_contextType.Load] (options {k=2;}: COMMA t+=test[expr_contextType.Load])*
-         (trailcomma=COMMA)?
+       t+=test[expr_contextType.Load] (options {k=2;}: COMMA t+=test[expr_contextType.Load])* (trailcomma=COMMA)?
        { $elts=$t;
            if ($trailcomma == null) {
                $newline = true;
@@ -694,11 +732,12 @@
 
 //continue_stmt: 'continue'
 continue_stmt
-    : CONTINUE {
-        if (!$suite.isEmpty() && $suite::continueIllegal) {
-            errorHandler.error("'continue' not supported inside 'finally' clause", new PythonTree($continue_stmt.start));
-        }
-    }
+    : CONTINUE
+      {
+          if (!$suite.isEmpty() && $suite::continueIllegal) {
+              errorHandler.error("'continue' not supported inside 'finally' clause", new PythonTree($continue_stmt.start));
+          }
+      }
    -> ^(CONTINUE<Continue>[$CONTINUE])
     ;
 
@@ -753,7 +792,8 @@
     ;
 
 //import_as_names: import_as_name (',' import_as_name)* [',']
-import_as_names returns [List<alias> atypes]
+import_as_names
+    returns [List<alias> atypes]
     : n+=import_as_name (COMMA! n+=import_as_name)*
     {
         $atypes = $n;
@@ -761,7 +801,8 @@
     ;
 
 //import_as_name: NAME [('as' | NAME) NAME]
-import_as_name returns [alias atype]
+import_as_name
+    returns [alias atype]
 @after {
     $import_as_name.tree = $atype;
 }
@@ -773,7 +814,8 @@
 
 //XXX: when does CPython Grammar match "dotted_name NAME NAME"?
 //dotted_as_name: dotted_name [('as' | NAME) NAME]
-dotted_as_name returns [alias atype]
+dotted_as_name
+    returns [alias atype]
 @after {
     $dotted_as_name.tree = $atype;
 }
@@ -785,7 +827,8 @@
     ;
 
 //dotted_as_names: dotted_as_name (',' dotted_as_name)*
-dotted_as_names returns [List<alias> atypes]
+dotted_as_names
+    returns [List<alias> atypes]
     : d+=dotted_as_name (COMMA! d+=dotted_as_name)*
     {
         $atypes = $d;
@@ -793,8 +836,10 @@
     ;
 
 //dotted_name: NAME ('.' NAME)*
-dotted_name returns [String name]
-    : NAME (DOT dn+=attr)* {
+dotted_name
+    returns [String name]
+    : NAME (DOT dn+=attr)*
+    {
         $name = actions.makeDottedText($NAME, $dn);
     }
     ;
@@ -813,8 +858,7 @@
 @after {
    $exec_stmt.tree = stype;
 }
-    : EXEC expr[expr_contextType.Load] (IN t1=test[expr_contextType.Load]
-        (COMMA t2=test[expr_contextType.Load])?)?
+    : EXEC expr[expr_contextType.Load] (IN t1=test[expr_contextType.Load] (COMMA t2=test[expr_contextType.Load])?)?
       {
          stype = new Exec($EXEC, actions.castExpr($expr.tree), actions.castExpr($t1.tree), actions.castExpr($t2.tree));
       }
@@ -845,10 +889,12 @@
     ;
 
 //not in CPython's Grammar file
-elif_clause [Token iftest] returns [List stypes]
-    : else_clause {
-        $stypes = $else_clause.stypes;
-    }
+elif_clause
+    [Token iftest] returns [List stypes]
+    : else_clause
+      {
+          $stypes = $else_clause.stypes;
+      }
     | ELIF test[expr_contextType.Load] COLON suite[false]
         (e2=elif_clause[$iftest]
        -> ^(ELIF<If>[$iftest, actions.castExpr($test.tree), actions.castStmts($suite.stypes), actions.makeElse($e2.stypes, $e2.tree)])
@@ -858,10 +904,12 @@
     ;
 
 //not in CPython's Grammar file
-else_clause returns [List stypes]
-    : ORELSE COLON elsesuite=suite[false] {
-        $stypes = $suite.stypes;
-    }
+else_clause
+    returns [List stypes]
+    : ORELSE COLON elsesuite=suite[false]
+      {
+          $stypes = $suite.stypes;
+      }
     ;
 
 //while_stmt: 'while' test ':' suite ['else' ':' suite]
@@ -873,9 +921,9 @@
    $while_stmt.tree = stype;
 }
     : WHILE test[expr_contextType.Load] COLON s1=suite[false] (ORELSE COLON s2=suite[false])?
-    {
-        stype = actions.makeWhile($WHILE, actions.castExpr($test.tree), $s1.stypes, $s2.stypes);
-    }
+      {
+          stype = actions.makeWhile($WHILE, actions.castExpr($test.tree), $s1.stypes, $s2.stypes);
+      }
     ;
 
 //for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]
@@ -933,7 +981,8 @@
     ;
 
 //with_var: ('as' | NAME) expr
-with_var returns [expr etype]
+with_var
+    returns [expr etype]
     : (AS | NAME) expr[expr_contextType.Store]
       {
           $etype = actions.castExpr($expr.tree);
@@ -948,7 +997,8 @@
     ;
 
 //suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT
-suite[boolean fromFinally] returns [List stypes]
+suite
+    [boolean fromFinally] returns [List stypes]
 scope {
     boolean continueIllegal;
 }
@@ -965,11 +1015,12 @@
           $stypes = $simple_stmt.stypes;
       }
     | NEWLINE INDENT
-      (stmt {
-          if ($stmt.stypes != null) {
-              $stypes.addAll($stmt.stypes);
-          }
-      }
+      (stmt
+       {
+           if ($stmt.stypes != null) {
+               $stypes.addAll($stmt.stypes);
+           }
+       }
       )+ DEDENT
     ;
 
@@ -1033,7 +1084,10 @@
     }
 }
     : left=expr[ctype]
-       ( ( comp_op right+=expr[ctype] {cmps.add($comp_op.op);}
+       ( ( comp_op right+=expr[ctype]
+           {
+               cmps.add($comp_op.op);
+           }
          )+
        |
       -> $left
@@ -1041,18 +1095,30 @@
     ;
 
 //comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not'
-comp_op returns [cmpopType op]
-    : LESS {$op = cmpopType.Lt;}
-    | GREATER {$op = cmpopType.Gt;}
-    | EQUAL {$op = cmpopType.Eq;}
-    | GREATEREQUAL {$op = cmpopType.GtE;}
-    | LESSEQUAL {$op = cmpopType.LtE;}
-    | ALT_NOTEQUAL {$op = cmpopType.NotEq;}
-    | NOTEQUAL {$op = cmpopType.NotEq;}
-    | IN {$op = cmpopType.In;}
-    | NOT IN {$op = cmpopType.NotIn;}
-    | IS {$op = cmpopType.Is;}
-    | IS NOT {$op = cmpopType.IsNot;}
+comp_op
+    returns [cmpopType op]
+    : LESS
+        {$op = cmpopType.Lt;}
+    | GREATER
+        {$op = cmpopType.Gt;}
+    | EQUAL
+        {$op = cmpopType.Eq;}
+    | GREATEREQUAL
+        {$op = cmpopType.GtE;}
+    | LESSEQUAL
+        {$op = cmpopType.LtE;}
+    | ALT_NOTEQUAL
+        {$op = cmpopType.NotEq;}
+    | NOTEQUAL
+        {$op = cmpopType.NotEq;}
+    | IN
+        {$op = cmpopType.In;}
+    | NOT IN
+        {$op = cmpopType.NotIn;}
+    | IS
+        {$op = cmpopType.Is;}
+    | IS NOT
+        {$op = cmpopType.IsNot;}
     ;
 
 
@@ -1119,16 +1185,22 @@
     }
 }
     : left=arith_expr
-        ( ( shift_op right+=arith_expr {ops.add($shift_op.op);}
+        ( ( shift_op right+=arith_expr
+            {
+                ops.add($shift_op.op);
+            }
           )+
         |
        -> $left
         )
     ;
 
-shift_op returns [operatorType op]
-    : LEFTSHIFT {$op = operatorType.LShift;}
-    | RIGHTSHIFT {$op = operatorType.RShift;}
+shift_op
+    returns [operatorType op]
+    : LEFTSHIFT
+        {$op = operatorType.LShift;}
+    | RIGHTSHIFT
+        {$op = operatorType.RShift;}
     ;
 
 //arith_expr: term (('+'|'-') term)*
@@ -1142,7 +1214,10 @@
     }
 }
     : left=term
-        ( (arith_op right+=term {ops.add($arith_op.op);}
+        ( (arith_op right+=term
+           {
+               ops.add($arith_op.op);
+           }
           )+
         |
        -> $left
@@ -1157,9 +1232,12 @@
         errorHandler.error("Internal Parser Error", badNode);
     }
 
-arith_op returns [operatorType op]
-    : PLUS {$op = operatorType.Add;}
-    | MINUS {$op = operatorType.Sub;}
+arith_op
+    returns [operatorType op]
+    : PLUS
+        {$op = operatorType.Add;}
+    | MINUS
+        {$op = operatorType.Sub;}
     ;
 
 //term: factor (('*'|'/'|'%'|'//') factor)*
@@ -1173,33 +1251,47 @@
     }
 }
     : left=factor
-        ( (term_op right+=factor {ops.add($term_op.op);}
+        ( (term_op right+=factor
+          {
+              ops.add($term_op.op);
+          }
           )+
         |
        -> $left
         )
     ;
 
-term_op returns [operatorType op]
-    :STAR {$op = operatorType.Mult;}
-    |SLASH {$op = operatorType.Div;}
-    |PERCENT {$op = operatorType.Mod;}
-    |DOUBLESLASH {$op = operatorType.FloorDiv;}
+term_op
+    returns [operatorType op]
+    :STAR
+        {$op = operatorType.Mult;}
+    |SLASH
+        {$op = operatorType.Div;}
+    |PERCENT
+        {$op = operatorType.Mod;}
+    |DOUBLESLASH
+        {$op = operatorType.FloorDiv;}
     ;
 
 //factor: ('+'|'-'|'~') factor | power
-factor returns [expr etype]
+factor
+    returns [expr etype]
 @after {
     $factor.tree = $etype;
 }
-    : PLUS p=factor {$etype = new UnaryOp($PLUS, unaryopType.UAdd, $p.etype);}
-    | MINUS m=factor {$etype = actions.negate($MINUS, $m.etype);}
-    | TILDE t=factor {$etype = new UnaryOp($TILDE, unaryopType.Invert, $t.etype);}
-    | power {$etype = actions.castExpr($power.tree);}
+    : PLUS p=factor
+        {$etype = new UnaryOp($PLUS, unaryopType.UAdd, $p.etype);}
+    | MINUS m=factor
+        {$etype = actions.negate($MINUS, $m.etype);}
+    | TILDE t=factor
+        {$etype = new UnaryOp($TILDE, unaryopType.Invert, $t.etype);}
+    | power
+        {$etype = actions.castExpr($power.tree);}
     ;
 
 //power: atom trailer* ['**' factor]
-power returns [expr etype]
+power
+    returns [expr etype]
 @after {
     $power.tree = $etype;
 }
@@ -1293,15 +1385,15 @@
 }
     : t+=test[$expr::ctype]
         (list_for[gens]
-          {
-              Collections.reverse(gens);
-              List<comprehension> c = gens;
-              etype = new ListComp($listmaker.start, actions.castExpr($t.get(0)), c);
-          }
+         {
+             Collections.reverse(gens);
+             List<comprehension> c = gens;
+             etype = new ListComp($listmaker.start, actions.castExpr($t.get(0)), c);
+         }
         | (options {greedy=true;}:COMMA t+=test[$expr::ctype])*
-          {
-              etype = new org.python.antlr.ast.List($lbrack, actions.castExprs($t), $expr::ctype);
-          }
+           {
+               etype = new org.python.antlr.ast.List($lbrack, actions.castExprs($t), $expr::ctype);
+           }
         ) (COMMA)?
     ;
 
@@ -1385,7 +1477,8 @@
     ;
 
 //subscript: '.' '.' '.' | test | [test] ':' [test] [sliceop]
-subscript returns [slice sltype]
+subscript
+    returns [slice sltype]
 @after {
     if ($sltype != null) {
         $subscript.tree = $sltype;
@@ -1410,17 +1503,20 @@
 //sliceop: ':' [test]
 sliceop
     : COLON
-     (test[expr_contextType.Load] -> test
-     |-> ^(COLON<Name>[$COLON, "None", expr_contextType.Load])
+     (test[expr_contextType.Load]
+    -> test
+     |
+    -> ^(COLON<Name>[$COLON, "None", expr_contextType.Load])
      )
     ;
 
 //exprlist: expr (',' expr)* [',']
-exprlist[expr_contextType ctype] returns [expr etype]
+exprlist
+    [expr_contextType ctype] returns [expr etype]
     : (expr[null] COMMA) => e+=expr[ctype] (options {k=2;}: COMMA e+=expr[ctype])* (COMMA)?
-      {
-          $etype = new Tuple($exprlist.start, actions.castExprs($e), ctype);
-      }
+       {
+           $etype = new Tuple($exprlist.start, actions.castExprs($e), ctype);
+       }
     | expr[ctype]
       {
         $etype = actions.castExpr($expr.tree);
@@ -1429,7 +1525,8 @@
 
 //not in CPython's Grammar file
 //Needed as an exprlist that does not produce tuples for del_stmt.
-del_list returns [List<expr> etypes]
+del_list
+    returns [List<expr> etypes]
     : e+=expr[expr_contextType.Del] (options {k=2;}: COMMA e+=expr[expr_contextType.Del])* (COMMA)?
       {
           $etypes = actions.makeDeleteList($e);
@@ -1445,7 +1542,8 @@
     ;
 
 //dictmaker: test ':' test (',' test ':' test)* [',']
-dictmaker returns [List keys, List values]
+dictmaker
+    returns [List keys, List values]
     : k+=test[expr_contextType.Load] COLON v+=test[expr_contextType.Load]
         (options {k=2;}:COMMA k+=test[expr_contextType.Load] COLON v+=test[expr_contextType.Load])*
         (COMMA)?
@@ -1477,7 +1575,8 @@
     ;
 
 //arglist: (argument ',')* (argument [',']| '*' test [',' '**' test] | '**' test)
-arglist returns [List args, List keywords, expr starargs, expr kwargs]
+arglist
+    returns [List args, List keywords, expr starargs, expr kwargs]
 @init {
     List arguments = new ArrayList();
     List kws = new ArrayList();
@@ -1510,7 +1609,8 @@
     ;
 
 //argument: test [gen_for] | test '=' test  # Really [keyword '='] test
-argument[List arguments, List kws, List gens, boolean first] returns [boolean genarg]
+argument
+    [List arguments, List kws, List gens, boolean first] returns [boolean genarg]
     : t1=test[expr_contextType.Load]
         ((ASSIGN t2=test[expr_contextType.Load])
           {
@@ -1529,7 +1629,8 @@
               List<comprehension> c = $gens;
               arguments.add(new GeneratorExp($t1.start, actions.castExpr($t1.tree), c));
           }
-        | {
+        |
+          {
               if (kws.size() > 0) {
                   errorHandler.error("non-keyword arg after keyword arg", $t1.tree);
               }
@@ -1559,9 +1660,9 @@
 //list_if: 'if' test [list_iter]
 list_if[List gens, List ifs]
     : IF test[expr_contextType.Load] (list_iter[gens, ifs])?
-    {
+      {
         ifs.add(actions.castExpr($test.tree));
-    }
+      }
     ;
 
 //gen_iter: gen_for | gen_if
@@ -1585,9 +1686,9 @@
 //gen_if: 'if' old_test [gen_iter]
 gen_if[List gens, List ifs]
     : IF test[expr_contextType.Load] gen_iter[gens, ifs]?
-    {
+      {
         ifs.add(actions.castExpr($test.tree));
-    }
+      }
     ;
 
 //yield_expr: 'yield' [testlist]


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6514] trunk/jython

[<fwi...@us...>]

Revision: 6514
          http://jython.svn.sourceforge.net/jython/?rev=6514&view=rev
Author:   fwierzbicki
Date:     2009-07-06 16:31:40 +0000 (Mon, 06 Jul 2009)

Log Message:
-----------
Tightened interactive parse of partial strings.  Two of the commented out tests
for test_codeop now pass.  Unfortunately one new test fails, but it is less
serious than the two that are now fixed, so commented it out for now.

Modified Paths:
--------------
    trunk/jython/Lib/test/test_codeop.py
    trunk/jython/grammar/PythonPartial.g

Modified: trunk/jython/Lib/test/test_codeop.py
===================================================================
--- trunk/jython/Lib/test/test_codeop.py	2009-07-06 14:15:32 UTC (rev 6513)
+++ trunk/jython/Lib/test/test_codeop.py	2009-07-06 16:31:40 UTC (rev 6514)
@@ -138,7 +138,7 @@
         ai("\n\ndef x():\n  pass")
 
         ai("a = 9+ \\")
-        #ai("a = 'a\\")
+        ai("a = 'a\\")
         ai("a = '''xy")
 
         ai("","eval")
@@ -147,7 +147,7 @@
         ai("(\n\n\n","eval")
         ai("(9+","eval")
         ai("9+ \\","eval")
-        #ai("lambda z: \\","eval")
+        ai("lambda z: \\","eval")
 
         #Did not work in Jython 2.5rc2 see first issue in
         # http://bugs.jython.org/issue1354
@@ -170,7 +170,7 @@
 
         #ai("a = 9+ \\\n")
         ai("a = 'a\\ ")
-        ai("a = 'a\\\n")
+        #ai("a = 'a\\\n")
 
         ai("a = 1","eval")
         #ai("a = (","eval")

Modified: trunk/jython/grammar/PythonPartial.g
===================================================================
--- trunk/jython/grammar/PythonPartial.g	2009-07-06 14:15:32 UTC (rev 6513)
+++ trunk/jython/grammar/PythonPartial.g	2009-07-06 16:31:40 UTC (rev 6514)
@@ -718,7 +718,8 @@
      | FLOAT
      | COMPLEX
      | (STRING)+
-     | STRINGPART
+     | TRISTRINGPART
+     | STRINGPART TRAILBACKSLASH
      ;
 
 //listmaker: test ( list_for | (',' test)* [','] )
@@ -1043,15 +1044,21 @@
         }
     ;
 
-STRINGPART
+TRISTRINGPART
     : ('r'|'u'|'ur'|'R'|'U'|'UR'|'uR'|'Ur')?
         (   '\'\'\'' ~('\'\'\'')*
         |   '"""' ~('"""')*
-        |   '"' (ESC|~('\\'|'\n'|'"'))* CONTINUED_LINE
+        )
+    ;
+
+STRINGPART
+    : ('r'|'u'|'ur'|'R'|'U'|'UR'|'uR'|'Ur')?
+        (   '"' (ESC|~('\\'|'\n'|'"'))* CONTINUED_LINE
         |   '\'' (ESC|~('\\'|'\n'|'\''))* CONTINUED_LINE
         )
     ;
 
+
 /** the two '"'? cause a warning -- is there a way to avoid that? */
 fragment
 TRIQUOTE


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6513] trunk/jython/NEWS

[<fwi...@us...>]

Revision: 6513
          http://jython.svn.sourceforge.net/jython/?rev=6513&view=rev
Author:   fwierzbicki
Date:     2009-07-06 14:15:32 +0000 (Mon, 06 Jul 2009)

Log Message:
-----------
The grammar cleanup in the last few checkins also fixed
http://bugs.jython.org/issue1366 "parsing of lamda expression fails" where
certain lambda expressions failed to parse in interactive mode.

Modified Paths:
--------------
    trunk/jython/NEWS

Modified: trunk/jython/NEWS
===================================================================
--- trunk/jython/NEWS	2009-07-06 13:47:19 UTC (rev 6512)
+++ trunk/jython/NEWS	2009-07-06 14:15:32 UTC (rev 6513)
@@ -5,6 +5,7 @@
     - Upgraded to ANTLR 3.1.3
     - [ 1859477 ] Dynamically loaded ServletFilters like PyServlet
   Bugs Fixed
+    - [ 1366 ] parsing of lamda expression fails
     - [ 1365 ] continuation lines fail in interactive interpreter
     - [ 1377 ] Event names shadowed by a field name on Java types leads to a NPE
     - [ 1381 ] Redundant declarations of interface implementation hides overriden methods


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6512] trunk/jython

[<fwi...@us...>]

Revision: 6512
          http://jython.svn.sourceforge.net/jython/?rev=6512&view=rev
Author:   fwierzbicki
Date:     2009-07-06 13:47:19 +0000 (Mon, 06 Jul 2009)

Log Message:
-----------
Fix for http://bugs.jython.org/issue1365 "continuation lines fail in
interactive interpreter".

Modified Paths:
--------------
    trunk/jython/NEWS
    trunk/jython/grammar/PythonPartial.g

Modified: trunk/jython/NEWS
===================================================================
--- trunk/jython/NEWS	2009-07-05 22:58:57 UTC (rev 6511)
+++ trunk/jython/NEWS	2009-07-06 13:47:19 UTC (rev 6512)
@@ -5,6 +5,7 @@
     - Upgraded to ANTLR 3.1.3
     - [ 1859477 ] Dynamically loaded ServletFilters like PyServlet
   Bugs Fixed
+    - [ 1365 ] continuation lines fail in interactive interpreter
     - [ 1377 ] Event names shadowed by a field name on Java types leads to a NPE
     - [ 1381 ] Redundant declarations of interface implementation hides overriden methods
 

Modified: trunk/jython/grammar/PythonPartial.g
===================================================================
--- trunk/jython/grammar/PythonPartial.g	2009-07-05 22:58:57 UTC (rev 6511)
+++ trunk/jython/grammar/PythonPartial.g	2009-07-06 13:47:19 UTC (rev 6512)
@@ -1047,6 +1047,8 @@
     : ('r'|'u'|'ur'|'R'|'U'|'UR'|'uR'|'Ur')?
         (   '\'\'\'' ~('\'\'\'')*
         |   '"""' ~('"""')*
+        |   '"' (ESC|~('\\'|'\n'|'"'))* CONTINUED_LINE
+        |   '\'' (ESC|~('\\'|'\n'|'\''))* CONTINUED_LINE
         )
     ;
 


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6511] trunk/jython/grammar/Python.g

[<fwi...@us...>]

Revision: 6511
          http://jython.svn.sourceforge.net/jython/?rev=6511&view=rev
Author:   fwierzbicki
Date:     2009-07-05 22:58:57 +0000 (Sun, 05 Jul 2009)

Log Message:
-----------
Now that PythonPartial has a separate Lexer, remove behavior related to
variable "partial".

Modified Paths:
--------------
    trunk/jython/grammar/Python.g

Modified: trunk/jython/grammar/Python.g
===================================================================
--- trunk/jython/grammar/Python.g	2009-07-05 22:34:44 UTC (rev 6510)
+++ trunk/jython/grammar/Python.g	2009-07-05 22:58:57 UTC (rev 6511)
@@ -1772,13 +1772,6 @@
         }
     ;
 
-STRINGPART
-    : {partial}?=> ('r'|'u'|'ur'|'R'|'U'|'UR'|'uR'|'Ur')?
-        (   '\'\'\'' ~('\'\'\'')*
-        |   '"""' ~('"""')*
-        )
-    ;
-
 /** the two '"'? cause a warning -- is there a way to avoid that? */
 fragment
 TRIQUOTE
@@ -1804,18 +1797,12 @@
     :    '\\' ('\r')? '\n' (' '|'\t')*  { $channel=HIDDEN; }
          ( c1=COMMENT
          | nl=NEWLINE {
-                          if (!partial) {
-                              emit(new CommonToken(NEWLINE,nl.getText()));
-                          }
+                          emit(new CommonToken(NEWLINE,nl.getText()));
                       }
          |
          ) {
                if (input.LA(1) == -1) {
-                   if (partial) {
-                       emit(new CommonToken(TRAILBACKSLASH,"\\"));
-                   } else {
-                       throw new ParseException("unexpected character after line continuation character");
-                   }
+                   throw new ParseException("unexpected character after line continuation character");
                }
            }
     ;


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6510] trunk/jython/grammar

[<fwi...@us...>]

Revision: 6510
          http://jython.svn.sourceforge.net/jython/?rev=6510&view=rev
Author:   fwierzbicki
Date:     2009-07-05 22:34:44 +0000 (Sun, 05 Jul 2009)

Log Message:
-----------
Whitespace fixes for easier diffs.

Modified Paths:
--------------
    trunk/jython/grammar/Python.g
    trunk/jython/grammar/PythonPartial.g

Modified: trunk/jython/grammar/Python.g
===================================================================
--- trunk/jython/grammar/Python.g	2009-07-05 18:07:53 UTC (rev 6509)
+++ trunk/jython/grammar/Python.g	2009-07-05 22:34:44 UTC (rev 6510)
@@ -156,7 +156,7 @@
 import java.util.Collections;
 import java.util.Iterator;
 import java.util.ListIterator;
-} 
+}
 
 @members {
     private ErrorHandler errorHandler;
@@ -195,7 +195,7 @@
 }
 }
 
-@lexer::header { 
+@lexer::header {
 package org.python.antlr;
 }
 
@@ -221,7 +221,7 @@
         this.errorHandler = eh;
     }
 
-    /** 
+    /**
      *  Taken directly from antlr's Lexer.java -- needs to be re-integrated every time
      *  we upgrade from Antlr (need to consider a Lexer subclass, though the issue would
      *  remain).
@@ -401,7 +401,7 @@
 @after {
    $decorator.tree = $etype;
 }
-    : AT dotted_attr 
+    : AT dotted_attr
     ( LPAREN
       ( arglist
         {
@@ -443,8 +443,10 @@
 
 //parameters: '(' [varargslist] ')'
 parameters returns [arguments args]
-    : LPAREN 
-      (varargslist {$args = $varargslist.args;}
+    : LPAREN
+      (varargslist
+          {$args = $varargslist.args;
+          }
       | { $args = new arguments($parameters.start, new ArrayList<expr>(), null, null, new ArrayList<expr>());
         }
       )
@@ -498,7 +500,7 @@
 @after {
     actions.checkAssign(actions.castExpr($fpdef.tree));
 }
-    : NAME 
+    : NAME
    -> ^(NAME<Name>[$NAME, $NAME.text, ctype])
     | (LPAREN fpdef[null] COMMA) => LPAREN fplist RPAREN
    -> ^(LPAREN<Tuple>[$fplist.start, actions.castExprs($fplist.etypes), expr_contextType.Store])
@@ -551,7 +553,7 @@
 
 //expr_stmt: testlist (augassign (yield_expr|testlist) |
 //                     ('=' (yield_expr|testlist))*)
-expr_stmt 
+expr_stmt
 @init {
     stmt stype = null;
 }
@@ -561,7 +563,7 @@
     }
 }
     : ((testlist[null] augassign) => lhs=testlist[expr_contextType.AugStore]
-        ( (aay=augassign y1=yield_expr 
+        ( (aay=augassign y1=yield_expr
             {
                 actions.checkAssign(actions.castExpr($lhs.tree));
                 stype = new AugAssign($lhs.tree, actions.castExpr($lhs.tree), $aay.op, actions.castExpr($y1.tree));
@@ -612,7 +614,7 @@
 //print_stmt: 'print' ( [ test (',' test)* [','] ] |
 //                      '>>' test [ (',' test)+ [','] ] )
 print_stmt
-    : PRINT 
+    : PRINT
       (t1=printlist
      -> ^(PRINT<Print>[$PRINT, null, actions.castExprs($t1.elts), $t1.newline])
       | RIGHTSHIFT t2=printlist2
@@ -702,7 +704,7 @@
 
 //return_stmt: 'return' [testlist]
 return_stmt
-    : RETURN 
+    : RETURN
       (testlist[expr_contextType.Load]
      -> ^(RETURN<Return>[$RETURN, actions.castExpr($testlist.tree)])
       |
@@ -737,7 +739,7 @@
 //import_from: ('from' ('.'* dotted_name | '.'+)
 //              'import' ('*' | '(' import_as_names ')' | import_as_names))
 import_from
-    : FROM (d+=DOT* dotted_name | d+=DOT+) IMPORT 
+    : FROM (d+=DOT* dotted_name | d+=DOT+) IMPORT
         (STAR
        -> ^(FROM<ImportFrom>[$FROM, actions.makeFromText($d, $dotted_name.name),
              actions.makeStarAlias($STAR), actions.makeLevel($d)])
@@ -1001,7 +1003,7 @@
 and_test[expr_contextType ctype]
 @after {
     if ($and != null) {
-        $and_test.tree = actions.makeBoolOp($left.tree, boolopType.And, $right); 
+        $and_test.tree = actions.makeBoolOp($left.tree, boolopType.And, $right);
     }
 }
     : left=not_test[ctype]
@@ -1064,7 +1066,7 @@
 }
 @after {
     if ($op != null) {
-        $expr.tree = actions.makeBinOp($left.tree, operatorType.BitOr, $right); 
+        $expr.tree = actions.makeBinOp($left.tree, operatorType.BitOr, $right);
     }
 }
     : left=xor_expr
@@ -1080,7 +1082,7 @@
 xor_expr
 @after {
     if ($op != null) {
-        $xor_expr.tree = actions.makeBinOp($left.tree, operatorType.BitXor, $right); 
+        $xor_expr.tree = actions.makeBinOp($left.tree, operatorType.BitXor, $right);
     }
 }
     : left=and_expr
@@ -1095,7 +1097,7 @@
 and_expr
 @after {
     if ($op != null) {
-        $and_expr.tree = actions.makeBinOp($left.tree, operatorType.BitAnd, $right); 
+        $and_expr.tree = actions.makeBinOp($left.tree, operatorType.BitAnd, $right);
     }
 }
     : left=shift_expr
@@ -1113,7 +1115,7 @@
 }
 @after {
     if (!ops.isEmpty()) {
-        $shift_expr.tree = actions.makeBinOp($left.tree, ops, $right); 
+        $shift_expr.tree = actions.makeBinOp($left.tree, ops, $right);
     }
 }
     : left=arith_expr
@@ -1240,7 +1242,7 @@
 //       '`' testlist1 '`' |
 //       NAME | NUMBER | STRING+)
 atom
-    : LPAREN 
+    : LPAREN
       ( yield_expr
      -> yield_expr
       | testlist_gexp
@@ -1256,7 +1258,7 @@
      -> ^(LBRACK<org.python.antlr.ast.List>[$LBRACK, new ArrayList<expr>(), $expr::ctype])
       )
       RBRACK
-    | LCURLY 
+    | LCURLY
        (dictmaker
       -> ^(LCURLY<Dict>[$LCURLY, actions.castExprs($dictmaker.keys),
               actions.castExprs($dictmaker.values)])
@@ -1276,7 +1278,7 @@
     -> ^(FLOAT<Num>[$FLOAT, actions.makeFloat($FLOAT)])
      | COMPLEX
     -> ^(COMPLEX<Num>[$COMPLEX, actions.makeComplex($COMPLEX)])
-     | (S+=STRING)+ 
+     | (S+=STRING)+
     -> ^(STRING<Str>[actions.extractStringToken($S), actions.extractStrings($S, encoding)])
      ;
 
@@ -1289,7 +1291,7 @@
 @after {
    $listmaker.tree = etype;
 }
-    : t+=test[$expr::ctype] 
+    : t+=test[$expr::ctype]
         (list_for[gens]
           {
               Collections.reverse(gens);
@@ -1301,7 +1303,7 @@
               etype = new org.python.antlr.ast.List($lbrack, actions.castExprs($t), $expr::ctype);
           }
         ) (COMMA)?
-          ;
+    ;
 
 //testlist_gexp: test ( gen_for | (',' test)* [','] )
 testlist_gexp
@@ -1354,7 +1356,7 @@
 
 //trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME
 trailer [Token begin, PythonTree tree]
-    : LPAREN 
+    : LPAREN
         (arglist
        -> ^(LPAREN<Call>[$begin, actions.castExpr($tree), actions.castExprs($arglist.args),
                actions.makeKeywords($arglist.keywords), $arglist.starargs, $arglist.kwargs])
@@ -1836,7 +1838,7 @@
 
 WS  :    {startPos>0}?=> (' '|'\t'|'\u000C')+ {$channel=HIDDEN;}
     ;
-    
+
 /** Grab everything before a real symbol.  Then if newline, kill it
  *  as this is a blank line.  If whitespace followed by comment, kill it
  *  as it's a comment on a line by itself.
@@ -1906,7 +1908,7 @@
 
     Only match \n here if we didn't start on left edge; let NEWLINE return that.
     Kill if newlines if we live on a line by ourselves
-    
+
     Consume any leading whitespace if it starts on left edge.
  */
 COMMENT

Modified: trunk/jython/grammar/PythonPartial.g
===================================================================
--- trunk/jython/grammar/PythonPartial.g	2009-07-05 18:07:53 UTC (rev 6509)
+++ trunk/jython/grammar/PythonPartial.g	2009-07-05 22:34:44 UTC (rev 6510)
@@ -60,14 +60,13 @@
  */
 
 grammar PythonPartial;
-
 options {
     tokenVocab=Python;
 }
 
-@header { 
+@header {
 package org.python.antlr;
-} 
+}
 
 @members {
     private ErrorHandler errorHandler = new FailFastHandler();
@@ -90,7 +89,7 @@
 }
 }
 
-@lexer::header { 
+@lexer::header {
 package org.python.antlr;
 }
 
@@ -116,7 +115,7 @@
         this.errorHandler = eh;
     }
 
-    /** 
+    /**
      *  Taken directly from antlr's Lexer.java -- needs to be re-integrated every time
      *  we upgrade from Antlr (need to consider a Lexer subclass, though the issue would
      *  remain).
@@ -172,15 +171,14 @@
 
 //eval_input: testlist NEWLINE* ENDMARKER
 eval_input
-
     : LEADING_WS? (NEWLINE)* testlist? (NEWLINE)* EOF
     ;
 
 //not in CPython's Grammar file
-dotted_attr 
+dotted_attr
     : NAME
-      ( (DOT NAME)+ 
-      | 
+      ( (DOT NAME)+
+      |
       )
     ;
 
@@ -223,95 +221,77 @@
     ;
 
 //decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE
-decorator 
-
-    : AT dotted_attr 
+decorator
+    : AT dotted_attr
     ( LPAREN
       ( arglist
-        
-      | 
+      |
       )
       RPAREN
-    | 
+    |
     ) NEWLINE
     ;
 
 //decorators: decorator+
-decorators 
+decorators
     : decorator+
-      
     ;
 
 //funcdef: [decorators] 'def' NAME parameters ':' suite
 funcdef
-
     : decorators? DEF NAME parameters COLON suite
-    
     ;
 
 //parameters: '(' [varargslist] ')'
-parameters 
-    : LPAREN 
-      (varargslist 
-      | 
+parameters
+    : LPAREN
+      (varargslist
+      |
       )
       RPAREN
     ;
 
 //not in CPython's Grammar file
-defparameter 
-
+defparameter
     : fpdef (ASSIGN test)?
-      
     ;
 
 //varargslist: ((fpdef ['=' test] ',')*
 //              ('*' NAME [',' '**' NAME] | '**' NAME) |
 //              fpdef ['=' test] (',' fpdef ['=' test])* [','])
-varargslist 
-
+varargslist
     : defparameter (options {greedy=true;}:COMMA defparameter)*
       (COMMA
           (STAR NAME (COMMA DOUBLESTAR NAME)?
           | DOUBLESTAR NAME
           )?
       )?
-      
     | STAR NAME (COMMA DOUBLESTAR NAME)?
-      
     | DOUBLESTAR NAME
-      
     ;
 
 //fpdef: NAME | '(' fplist ')'
 fpdef
-    : NAME 
-   
+    : NAME
     | (LPAREN fpdef COMMA) => LPAREN fplist RPAREN
-   
     | LPAREN fplist RPAREN
-   
     ;
 
 //fplist: fpdef (',' fpdef)* [',']
-fplist 
+fplist
     : fpdef
       (options {greedy=true;}:COMMA fpdef)* (COMMA)?
-      
     ;
 
 //stmt: simple_stmt | compound_stmt
-stmt 
+stmt
     : simple_stmt
-      
     | compound_stmt
-      
     ;
 
 //simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE
-simple_stmt 
+simple_stmt
     : small_stmt (options {greedy=true;}:SEMI small_stmt)* (SEMI)? (NEWLINE|EOF)
-      
     ;
 
 //small_stmt: (expr_stmt | print_stmt  | del_stmt | pass_stmt | flow_stmt |
@@ -327,57 +307,48 @@
            | assert_stmt
            ;
 
-expr_stmt 
-
+expr_stmt
     : ((testlist augassign) => testlist
-        ( (augassign yield_expr 
-            
+        ( (augassign yield_expr
           )
         | (augassign testlist
-            
           )
         )
     | (testlist ASSIGN) => testlist
         (
         | ((ASSIGN testlist)+
-       
           )
         | ((ASSIGN yield_expr)+
-       
           )
         )
     | testlist
-      
     )
     ;
 
 //augassign: ('+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' |
 //            '<<=' | '>>=' | '**=' | '//=')
-augassign 
-    : PLUSEQUAL 
-    | MINUSEQUAL 
-    | STAREQUAL 
-    | SLASHEQUAL 
-    | PERCENTEQUAL 
-    | AMPEREQUAL 
-    | VBAREQUAL 
-    | CIRCUMFLEXEQUAL 
-    | LEFTSHIFTEQUAL 
-    | RIGHTSHIFTEQUAL 
-    | DOUBLESTAREQUAL 
-    | DOUBLESLASHEQUAL 
+augassign
+    : PLUSEQUAL
+    | MINUSEQUAL
+    | STAREQUAL
+    | SLASHEQUAL
+    | PERCENTEQUAL
+    | AMPEREQUAL
+    | VBAREQUAL
+    | CIRCUMFLEXEQUAL
+    | LEFTSHIFTEQUAL
+    | RIGHTSHIFTEQUAL
+    | DOUBLESTAREQUAL
+    | DOUBLESLASHEQUAL
     ;
 
 //print_stmt: 'print' ( [ test (',' test)* [','] ] |
 //                      '>>' test [ (',' test)+ [','] ] )
 print_stmt
-    : PRINT 
+    : PRINT
       (printlist
-     
       | RIGHTSHIFT printlist
-     
       |
-     
       )
       ;
 
@@ -389,13 +360,11 @@
 //del_stmt: 'del' exprlist
 del_stmt
     : DELETE exprlist
-   
     ;
 
 //pass_stmt: 'pass'
 pass_stmt
     : PASS
-   
     ;
 
 //flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt
@@ -410,35 +379,30 @@
 //break_stmt: 'break'
 break_stmt
     : BREAK
-   
     ;
 
 //continue_stmt: 'continue'
 continue_stmt
-    : CONTINUE 
-   
+    : CONTINUE
     ;
 
 //return_stmt: 'return' [testlist]
 return_stmt
-    : RETURN 
+    : RETURN
       (testlist
-     
       |
-     
       )
       ;
 
 //yield_stmt: yield_expr
 yield_stmt
-    : yield_expr 
+    : yield_expr
     ;
 
 //raise_stmt: 'raise' [test [',' test [',' test]]]
 raise_stmt
     : RAISE (test (COMMA test
         (COMMA test)?)?)?
-   
     ;
 
 //import_stmt: import_name | import_from
@@ -450,73 +414,58 @@
 //import_name: 'import' dotted_as_names
 import_name
     : IMPORT dotted_as_names
-   
     ;
 
 //import_from: ('from' ('.'* dotted_name | '.'+)
 //              'import' ('*' | '(' import_as_names ')' | import_as_names))
 import_from
-    : FROM (DOT* dotted_name | DOT+) IMPORT 
+    : FROM (DOT* dotted_name | DOT+) IMPORT
         (STAR
-       
         | import_as_names
-       
         | LPAREN import_as_names COMMA? RPAREN
-       
         )
     ;
 
 //import_as_names: import_as_name (',' import_as_name)* [',']
-import_as_names 
+import_as_names
     : import_as_name (COMMA import_as_name)*
-    
     ;
 
 //import_as_name: NAME [('as' | NAME) NAME]
-import_as_name 
-
+import_as_name
     : NAME (AS NAME)?
-    
     ;
 
 //XXX: when does CPython Grammar match "dotted_name NAME NAME"?
 //dotted_as_name: dotted_name [('as' | NAME) NAME]
-dotted_as_name 
-
-
+dotted_as_name
     : dotted_name (AS NAME)?
-    
     ;
 
 //dotted_as_names: dotted_as_name (',' dotted_as_name)*
-dotted_as_names 
+dotted_as_names
     : dotted_as_name (COMMA dotted_as_name)*
-    
     ;
 
 //dotted_name: NAME ('.' NAME)*
-dotted_name 
-    : NAME (DOT attr)* 
+dotted_name
+    : NAME (DOT attr)*
     ;
 
 //global_stmt: 'global' NAME (',' NAME)*
 global_stmt
     : GLOBAL NAME (COMMA NAME)*
-   
     ;
 
 //exec_stmt: 'exec' expr ['in' test [',' test]]
 exec_stmt
-
     : EXEC expr (IN test
         (COMMA test)?)?
-      
     ;
 
 //assert_stmt: 'assert' test [',' test]
 assert_stmt
     : ASSERT test (COMMA test)?
-   
     ;
 
 //compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | funcdef | classdef
@@ -533,38 +482,31 @@
 //if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
 if_stmt
     : IF test COLON suite elif_clause?
-   
     ;
 
 //not in CPython's Grammar file
-elif_clause  
-    : else_clause 
+elif_clause
+    : else_clause
     | ELIF test COLON suite
         (elif_clause
-       
         |
-       
         )
     ;
 
 //not in CPython's Grammar file
-else_clause 
-    : ORELSE COLON suite 
+else_clause
+    : ORELSE COLON suite
     ;
 
 //while_stmt: 'while' test ':' suite ['else' ':' suite]
 while_stmt
-
     : WHILE test COLON suite (ORELSE COLON suite)?
-    
     ;
 
 //for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]
 for_stmt
-
     : FOR exprlist IN testlist COLON suite
         (ORELSE COLON suite)?
-      
     ;
 
 //try_stmt: ('try' ':' suite
@@ -573,39 +515,30 @@
 //           ['finally' ':' suite] |
 //           'finally' ':' suite))
 try_stmt
-
     : TRY COLON suite
       ( except_clause+ (ORELSE COLON suite)? (FINALLY COLON suite)?
-        
       | FINALLY COLON suite
-        
       )
       ;
 
 //with_stmt: 'with' test [ with_var ] ':' suite
 with_stmt
-
     : WITH test (with_var)? COLON suite
-      
     ;
 
 //with_var: ('as' | NAME) expr
-with_var 
+with_var
     : (AS | NAME) expr
-      
     ;
 
 //except_clause: 'except' [test [',' test]]
 except_clause
     : EXCEPT (test (COMMA test)?)? COLON suite
-   
     ;
 
 //suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT
-suite 
-
+suite
     : simple_stmt
-      
     | NEWLINE (EOF
               | (DEDENT)+ EOF
               |INDENT (stmt)+ (DEDENT
@@ -618,9 +551,7 @@
 test
     :or_test
       ( (IF or_test ORELSE) => IF or_test ORELSE test
-     
       |
-     
       )
     | lambdef
     ;
@@ -631,7 +562,6 @@
         ( (OR and_test
           )+
         |
-       
         )
     ;
 
@@ -641,141 +571,122 @@
         ( (AND not_test
           )+
         |
-       
         )
     ;
 
 //not_test: 'not' not_test | comparison
 not_test
     : NOT not_test
-   
     | comparison
     ;
 
 //comparison: expr (comp_op expr)*
 comparison
     : expr
-       ( ( comp_op expr 
+       ( ( comp_op expr
          )+
        |
-      
        )
     ;
 
 //comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not'
-comp_op 
-    : LESS 
-    | GREATER 
-    | EQUAL 
-    | GREATEREQUAL 
-    | LESSEQUAL 
-    | ALT_NOTEQUAL 
-    | NOTEQUAL 
-    | IN 
-    | NOT IN 
-    | IS 
-    | IS NOT 
+comp_op
+    : LESS
+    | GREATER
+    | EQUAL
+    | GREATEREQUAL
+    | LESSEQUAL
+    | ALT_NOTEQUAL
+    | NOTEQUAL
+    | IN
+    | NOT IN
+    | IS
+    | IS NOT
     ;
 
-
 //expr: xor_expr ('|' xor_expr)*
 expr
     : xor_expr
         ( (VBAR xor_expr
           )+
         |
-       
         )
     ;
 
-
 //xor_expr: and_expr ('^' and_expr)*
 xor_expr
-
     : and_expr
         ( (CIRCUMFLEX and_expr
           )+
         |
-       
         )
     ;
 
 //and_expr: shift_expr ('&' shift_expr)*
 and_expr
-
     : shift_expr
         ( (AMPER shift_expr
           )+
         |
-       
         )
     ;
 
 //shift_expr: arith_expr (('<<'|'>>') arith_expr)*
 shift_expr
-
     : arith_expr
-        ( ( shift_op arith_expr 
+        ( ( shift_op arith_expr
           )+
         |
-       
         )
     ;
 
-shift_op 
-    : LEFTSHIFT 
-    | RIGHTSHIFT 
+shift_op
+    : LEFTSHIFT
+    | RIGHTSHIFT
     ;
 
 //arith_expr: term (('+'|'-') term)*
 arith_expr
-
     : term
-        ( (arith_op term 
+        ( (arith_op term
           )+
         |
-       
         )
     ;
 
-arith_op 
-    : PLUS 
-    | MINUS 
+arith_op
+    : PLUS
+    | MINUS
     ;
 
 //term: factor (('*'|'/'|'%'|'//') factor)*
 term
-
     : factor
-        ( (term_op factor 
+        ( (term_op factor
           )+
         |
-       
         )
     ;
 
-term_op 
-    :STAR 
-    |SLASH 
-    |PERCENT 
-    |DOUBLESLASH 
+term_op
+    :STAR
+    |SLASH
+    |PERCENT
+    |DOUBLESLASH
     ;
 
 //factor: ('+'|'-'|'~') factor | power
-factor 
-
-    : PLUS factor 
-    | MINUS factor 
-    | TILDE factor 
-    | power 
+factor
+    : PLUS factor
+    | MINUS factor
+    | TILDE factor
+    | power
     | TRAILBACKSLASH
     ;
 
 //power: atom trailer* ['**' factor]
-power 
-
+power
     : atom (trailer)* (options {greedy=true;}:DOUBLESTAR factor)?
-      
     ;
 
 //atom: ('(' [yield_expr|testlist_gexp] ')' |
@@ -784,226 +695,176 @@
 //       '`' testlist1 '`' |
 //       NAME | NUMBER | STRING+)
 atom
-    : LPAREN 
+    : LPAREN
       ( yield_expr
-     
       | testlist_gexp
-     
       |
-     
       )
       RPAREN
     | LBRACK
       (listmaker
-     
       |
-     
       )
       RBRACK
-    | LCURLY 
+    | LCURLY
        (dictmaker
-      
        |
-      
        )
        RCURLY
      | BACKQUOTE testlist BACKQUOTE
-    
      | NAME
-    
      | INT
-    
      | LONGINT
-    
      | FLOAT
-    
      | COMPLEX
-    
-     | (STRING)+ 
-    
+     | (STRING)+
      | STRINGPART
      ;
 
 //listmaker: test ( list_for | (',' test)* [','] )
 listmaker
-    : test 
+    : test
         (list_for
-          
         | (options {greedy=true;}:COMMA test)*
-          
         ) (COMMA)?
-          ;
+    ;
 
 //testlist_gexp: test ( gen_for | (',' test)* [','] )
 testlist_gexp
-
     : test
         ( ((options {k=2;}: COMMA test)* (COMMA)?
-         
-         
           )
         | (gen_for
-           
           )
         )
     ;
 
 //lambdef: 'lambda' [varargslist] ':' test
 lambdef
-
     : LAMBDA (varargslist)? COLON test
-      
     ;
 
 //trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME
-trailer 
-    : LPAREN 
+trailer
+    : LPAREN
         (arglist
-       
         |
-       
         )
       RPAREN
     | LBRACK subscriptlist RBRACK
-   
     | DOT attr
-   
     ;
 
 //subscriptlist: subscript (',' subscript)* [',']
 subscriptlist
     : subscript (options {greedy=true;}:COMMA subscript)* (COMMA)?
-      
     ;
 
 //subscript: '.' '.' '.' | test | [test] ':' [test] [sliceop]
-subscript 
-
+subscript
     : DOT DOT DOT
-   
     | (test COLON)
    => test (COLON (test)? (sliceop)?)?
-      
     | (COLON)
    => COLON (test)? (sliceop)?
-      
     | test
-   
     ;
 
 //sliceop: ':' [test]
 sliceop
     : COLON
-     (test 
+     (test
      |
      )
     ;
 
 //exprlist: expr (',' expr)* [',']
-exprlist 
+exprlist
     : (expr COMMA) => expr (options {k=2;}: COMMA expr)* (COMMA)?
-      
     | expr
-      
     ;
 
 //not in CPython's Grammar file
 //Needed as an exprlist that does not produce tuples for del_stmt.
-del_list 
+del_list
     : expr (options {k=2;}: COMMA expr)* (COMMA)?
-      
     ;
 
 //testlist: test (',' test)* [',']
 testlist
     : (test COMMA)
    => test (options {k=2;}: COMMA test)* (COMMA)?
-   
     | test
     ;
 
 //dictmaker: test ':' test (',' test ':' test)* [',']
-dictmaker 
+dictmaker
     : test COLON test
         (options {k=2;}:COMMA test COLON test)*
         (COMMA)?
-      
     ;
 
 //classdef: 'class' NAME ['(' [testlist] ')'] ':' suite
 classdef
-
     : decorators? CLASS NAME (LPAREN testlist? RPAREN)? COLON suite
-      
     ;
 
 //arglist: (argument ',')* (argument [',']| '*' test [',' '**' test] | '**' test)
-arglist 
-
+arglist
     : argument (COMMA argument)*
           (COMMA
               ( STAR test (COMMA DOUBLESTAR test)?
               | DOUBLESTAR test
               )?
           )?
-      
     | STAR test (COMMA DOUBLESTAR test)?
-      
     | DOUBLESTAR test
-      
     ;
 
 //argument: test [gen_for] | test '=' test  # Really [keyword '='] test
-argument 
+argument
     : test
         ((ASSIGN test)
-          
         | gen_for
-          
-        | 
+        |
         )
     ;
 
 //list_iter: list_for | list_if
-list_iter 
+list_iter
     : list_for
     | list_if
     ;
 
 //list_for: 'for' exprlist 'in' testlist_safe [list_iter]
-list_for 
+list_for
     : FOR exprlist IN testlist (list_iter)?
-      
     ;
 
 //list_if: 'if' test [list_iter]
 list_if
     : IF test (list_iter)?
-    
     ;
 
 //gen_iter: gen_for | gen_if
-gen_iter 
+gen_iter
     : gen_for
     | gen_if
     ;
 
 //gen_for: 'for' exprlist 'in' or_test [gen_iter]
-gen_for 
+gen_for
     : FOR exprlist IN or_test gen_iter?
-      
     ;
 
 //gen_if: 'if' old_test [gen_iter]
 gen_if
     : IF test gen_iter?
-    
     ;
 
 //yield_expr: 'yield' [testlist]
 yield_expr
     : YIELD testlist?
-   
     ;
 
 AS        : 'as' ;
@@ -1240,7 +1101,7 @@
 
 WS  :    {startPos>0}?=> (' '|'\t'|'\u000C')+ {$channel=HIDDEN;}
     ;
-    
+
 /** Grab everything before a real symbol.  Then if newline, kill it
  *  as this is a blank line.  If whitespace followed by comment, kill it
  *  as it's a comment on a line by itself.
@@ -1310,7 +1171,7 @@
 
     Only match \n here if we didn't start on left edge; let NEWLINE return that.
     Kill if newlines if we live on a line by ourselves
-    
+
     Consume any leading whitespace if it starts on left edge.
  */
 COMMENT


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6509] trunk/jython

[<fwi...@us...>]

Revision: 6509
          http://jython.svn.sourceforge.net/jython/?rev=6509&view=rev
Author:   fwierzbicki
Date:     2009-07-05 18:07:53 +0000 (Sun, 05 Jul 2009)

Log Message:
-----------
PythonPartial now has it's own lexer.

Modified Paths:
--------------
    trunk/jython/grammar/PythonPartial.g
    trunk/jython/src/org/python/antlr/BaseParser.java
    trunk/jython/src/org/python/core/ParserFacade.java
    trunk/jython/tests/java/org/python/antlr/PythonPartialTester.java

Modified: trunk/jython/grammar/PythonPartial.g
===================================================================
--- trunk/jython/grammar/PythonPartial.g	2009-07-05 14:55:44 UTC (rev 6508)
+++ trunk/jython/grammar/PythonPartial.g	2009-07-05 18:07:53 UTC (rev 6509)
@@ -59,7 +59,7 @@
  *
  */
 
-parser grammar PythonPartial;
+grammar PythonPartial;
 
 options {
     tokenVocab=Python;
@@ -90,6 +90,78 @@
 }
 }
 
+@lexer::header { 
+package org.python.antlr;
+}
+
+@lexer::members {
+/** Handles context-sensitive lexing of implicit line joining such as
+ *  the case where newline is ignored in cases like this:
+ *  a = [3,
+ *       4]
+ */
+
+//For use in partial parsing.
+public boolean eofWhileNested = false;
+public boolean partial = false;
+
+int implicitLineJoiningLevel = 0;
+int startPos=-1;
+
+//If you want to use another error recovery mechanism change this
+//and the same one in the parser.
+private ErrorHandler errorHandler;
+
+    public void setErrorHandler(ErrorHandler eh) {
+        this.errorHandler = eh;
+    }
+
+    /** 
+     *  Taken directly from antlr's Lexer.java -- needs to be re-integrated every time
+     *  we upgrade from Antlr (need to consider a Lexer subclass, though the issue would
+     *  remain).
+     */
+    public Token nextToken() {
+        while (true) {
+            state.token = null;
+            state.channel = Token.DEFAULT_CHANNEL;
+            state.tokenStartCharIndex = input.index();
+            state.tokenStartCharPositionInLine = input.getCharPositionInLine();
+            state.tokenStartLine = input.getLine();
+            state.text = null;
+            if ( input.LA(1)==CharStream.EOF ) {
+                if (implicitLineJoiningLevel > 0) {
+                    eofWhileNested = true;
+                }
+                return Token.EOF_TOKEN;
+            }
+            try {
+                mTokens();
+                if ( state.token==null ) {
+                    emit();
+                }
+                else if ( state.token==Token.SKIP_TOKEN ) {
+                    continue;
+                }
+                return state.token;
+            } catch (NoViableAltException nva) {
+                errorHandler.reportError(this, nva);
+                errorHandler.recover(this, nva); // throw out current char and try again
+            } catch (FailedPredicateException fp) {
+                //XXX: added this for failed STRINGPART -- the FailedPredicateException
+                //     hides a NoViableAltException.  This should be the only
+                //     FailedPredicateException that gets thrown by the lexer.
+                errorHandler.reportError(this, fp);
+                errorHandler.recover(this, fp); // throw out current char and try again
+            } catch (RecognitionException re) {
+                errorHandler.reportError(this, re);
+                // match() routine has already called recover()
+            }
+        }
+    }
+}
+
+
 //single_input: NEWLINE | simple_stmt | compound_stmt NEWLINE
 single_input
 
@@ -934,3 +1006,318 @@
    
     ;
 
+AS        : 'as' ;
+ASSERT    : 'assert' ;
+BREAK     : 'break' ;
+CLASS     : 'class' ;
+CONTINUE  : 'continue' ;
+DEF       : 'def' ;
+DELETE    : 'del' ;
+ELIF      : 'elif' ;
+EXCEPT    : 'except' ;
+EXEC      : 'exec' ;
+FINALLY   : 'finally' ;
+FROM      : 'from' ;
+FOR       : 'for' ;
+GLOBAL    : 'global' ;
+IF        : 'if' ;
+IMPORT    : 'import' ;
+IN        : 'in' ;
+IS        : 'is' ;
+LAMBDA    : 'lambda' ;
+ORELSE    : 'else' ;
+PASS      : 'pass'  ;
+PRINT     : 'print' ;
+RAISE     : 'raise' ;
+RETURN    : 'return' ;
+TRY       : 'try' ;
+WHILE     : 'while' ;
+WITH      : 'with' ;
+YIELD     : 'yield' ;
+
+LPAREN    : '(' {implicitLineJoiningLevel++;} ;
+
+RPAREN    : ')' {implicitLineJoiningLevel--;} ;
+
+LBRACK    : '[' {implicitLineJoiningLevel++;} ;
+
+RBRACK    : ']' {implicitLineJoiningLevel--;} ;
+
+COLON     : ':' ;
+
+COMMA    : ',' ;
+
+SEMI    : ';' ;
+
+PLUS    : '+' ;
+
+MINUS    : '-' ;
+
+STAR    : '*' ;
+
+SLASH    : '/' ;
+
+VBAR    : '|' ;
+
+AMPER    : '&' ;
+
+LESS    : '<' ;
+
+GREATER    : '>' ;
+
+ASSIGN    : '=' ;
+
+PERCENT    : '%' ;
+
+BACKQUOTE    : '`' ;
+
+LCURLY    : '{' {implicitLineJoiningLevel++;} ;
+
+RCURLY    : '}' {implicitLineJoiningLevel--;} ;
+
+CIRCUMFLEX    : '^' ;
+
+TILDE    : '~' ;
+
+EQUAL    : '==' ;
+
+NOTEQUAL    : '!=' ;
+
+ALT_NOTEQUAL: '<>' ;
+
+LESSEQUAL    : '<=' ;
+
+LEFTSHIFT    : '<<' ;
+
+GREATEREQUAL    : '>=' ;
+
+RIGHTSHIFT    : '>>' ;
+
+PLUSEQUAL    : '+=' ;
+
+MINUSEQUAL    : '-=' ;
+
+DOUBLESTAR    : '**' ;
+
+STAREQUAL    : '*=' ;
+
+DOUBLESLASH    : '//' ;
+
+SLASHEQUAL    : '/=' ;
+
+VBAREQUAL    : '|=' ;
+
+PERCENTEQUAL    : '%=' ;
+
+AMPEREQUAL    : '&=' ;
+
+CIRCUMFLEXEQUAL    : '^=' ;
+
+LEFTSHIFTEQUAL    : '<<=' ;
+
+RIGHTSHIFTEQUAL    : '>>=' ;
+
+DOUBLESTAREQUAL    : '**=' ;
+
+DOUBLESLASHEQUAL    : '//=' ;
+
+DOT : '.' ;
+
+AT : '@' ;
+
+AND : 'and' ;
+
+OR : 'or' ;
+
+NOT : 'not' ;
+
+FLOAT
+    :   '.' DIGITS (Exponent)?
+    |   DIGITS '.' Exponent
+    |   DIGITS ('.' (DIGITS (Exponent)?)? | Exponent)
+    ;
+
+LONGINT
+    :   INT ('l'|'L')
+    ;
+
+fragment
+Exponent
+    :    ('e' | 'E') ( '+' | '-' )? DIGITS
+    ;
+
+INT :   // Hex
+        '0' ('x' | 'X') ( '0' .. '9' | 'a' .. 'f' | 'A' .. 'F' )+
+    |   // Octal
+        '0'  ( '0' .. '7' )*
+    |   '1'..'9' DIGITS*
+    ;
+
+COMPLEX
+    :   DIGITS+ ('j'|'J')
+    |   FLOAT ('j'|'J')
+    ;
+
+fragment
+DIGITS : ( '0' .. '9' )+ ;
+
+NAME:    ( 'a' .. 'z' | 'A' .. 'Z' | '_')
+        ( 'a' .. 'z' | 'A' .. 'Z' | '_' | '0' .. '9' )*
+    ;
+
+/** Match various string types.  Note that greedy=false implies '''
+ *  should make us exit loop not continue.
+ */
+STRING
+    :   ('r'|'u'|'ur'|'R'|'U'|'UR'|'uR'|'Ur')?
+        (   '\'\'\'' (options {greedy=false;}:TRIAPOS)* '\'\'\''
+        |   '"""' (options {greedy=false;}:TRIQUOTE)* '"""'
+        |   '"' (ESC|~('\\'|'\n'|'"'))* '"'
+        |   '\'' (ESC|~('\\'|'\n'|'\''))* '\''
+        ) {
+           if (state.tokenStartLine != input.getLine()) {
+               state.tokenStartLine = input.getLine();
+               state.tokenStartCharPositionInLine = -2;
+           }
+        }
+    ;
+
+STRINGPART
+    : ('r'|'u'|'ur'|'R'|'U'|'UR'|'uR'|'Ur')?
+        (   '\'\'\'' ~('\'\'\'')*
+        |   '"""' ~('"""')*
+        )
+    ;
+
+/** the two '"'? cause a warning -- is there a way to avoid that? */
+fragment
+TRIQUOTE
+    : '"'? '"'? (ESC|~('\\'|'"'))+
+    ;
+
+/** the two '\''? cause a warning -- is there a way to avoid that? */
+fragment
+TRIAPOS
+    : '\''? '\''? (ESC|~('\\'|'\''))+
+    ;
+
+fragment
+ESC
+    :    '\\' .
+    ;
+
+/** Consume a newline and any whitespace at start of next line
+ *  unless the next line contains only white space, in that case
+ *  emit a newline.
+ */
+CONTINUED_LINE
+    :    '\\' ('\r')? '\n' (' '|'\t')*  { $channel=HIDDEN; }
+         ( c1=COMMENT
+         | nl=NEWLINE
+         |
+         ) {
+               if (input.LA(1) == -1) {
+                   emit(new CommonToken(TRAILBACKSLASH,"\\"));
+               }
+           }
+    ;
+
+/** Treat a sequence of blank lines as a single blank line.  If
+ *  nested within a (..), {..}, or [..], then ignore newlines.
+ *  If the first newline starts in column one, they are to be ignored.
+ *
+ *  Frank Wierzbicki added: Also ignore FORMFEEDS (\u000C).
+ */
+NEWLINE
+@init {
+    int newlines = 0;
+}
+    :   (('\u000C')?('\r')? '\n' {newlines++; } )+ {
+         if ( startPos==0 || implicitLineJoiningLevel>0 )
+            $channel=HIDDEN;
+        }
+    ;
+
+WS  :    {startPos>0}?=> (' '|'\t'|'\u000C')+ {$channel=HIDDEN;}
+    ;
+    
+/** Grab everything before a real symbol.  Then if newline, kill it
+ *  as this is a blank line.  If whitespace followed by comment, kill it
+ *  as it's a comment on a line by itself.
+ *
+ *  Ignore leading whitespace when nested in [..], (..), {..}.
+ */
+LEADING_WS
+@init {
+    int spaces = 0;
+    int newlines = 0;
+}
+    :   {startPos==0}?=>
+        (   {implicitLineJoiningLevel>0}? ( ' ' | '\t' )+ {$channel=HIDDEN;}
+        |    (     ' '  { spaces++; }
+             |    '\t' { spaces += 8; spaces -= (spaces \% 8); }
+             )+
+             ( ('\r')? '\n' {newlines++; }
+             )* {
+                   if (input.LA(1) != -1 || newlines == 0) {
+                       // make a string of n spaces where n is column number - 1
+                       char[] indentation = new char[spaces];
+                       for (int i=0; i<spaces; i++) {
+                           indentation[i] = ' ';
+                       }
+                       CommonToken c = new CommonToken(LEADING_WS,new String(indentation));
+                       c.setLine(input.getLine());
+                       c.setCharPositionInLine(input.getCharPositionInLine());
+                       c.setStartIndex(input.index() - 1);
+                       c.setStopIndex(input.index() - 1);
+                       emit(c);
+                       // kill trailing newline if present and then ignore
+                       if (newlines != 0) {
+                           if (state.token!=null) {
+                               state.token.setChannel(HIDDEN);
+                           } else {
+                               $channel=HIDDEN;
+                           }
+                       }
+                   } else {
+                       // make a string of n newlines
+                       char[] nls = new char[newlines];
+                       for (int i=0; i<newlines; i++) {
+                           nls[i] = '\n';
+                       }
+                       CommonToken c = new CommonToken(NEWLINE,new String(nls));
+                       c.setLine(input.getLine());
+                       c.setCharPositionInLine(input.getCharPositionInLine());
+                       c.setStartIndex(input.index() - 1);
+                       c.setStopIndex(input.index() - 1);
+                       emit(c);
+                   }
+                }
+        )
+    ;
+
+/** Comments not on line by themselves are turned into newlines.
+
+    b = a # end of line comment
+
+    or
+
+    a = [1, # weird
+         2]
+
+    This rule is invoked directly by nextToken when the comment is in
+    first column or when comment is on end of nonwhitespace line.
+
+    Only match \n here if we didn't start on left edge; let NEWLINE return that.
+    Kill if newlines if we live on a line by ourselves
+    
+    Consume any leading whitespace if it starts on left edge.
+ */
+COMMENT
+@init {
+    $channel=HIDDEN;
+}
+    :    {startPos==0}?=> (' '|'\t')* '#' (~'\n')* '\n'+
+    |    '#' (~'\n')* // let NEWLINE handle \n unless char pos==0 for '#'
+    ;
+

Modified: trunk/jython/src/org/python/antlr/BaseParser.java
===================================================================
--- trunk/jython/src/org/python/antlr/BaseParser.java	2009-07-05 14:55:44 UTC (rev 6508)
+++ trunk/jython/src/org/python/antlr/BaseParser.java	2009-07-05 18:07:53 UTC (rev 6509)
@@ -41,7 +41,18 @@
             return super.nextToken();
         }
     }
-    
+
+    public static class PyPartialLexer extends PythonPartialLexer {
+        public PyPartialLexer(CharStream lexer) {
+            super(lexer);
+        }
+
+        public Token nextToken() {
+            startPos = getCharPositionInLine();
+            return super.nextToken();
+        }
+    }
+     
     private CharStream charStream(boolean single) {
         return charStream;
     }

Modified: trunk/jython/src/org/python/core/ParserFacade.java
===================================================================
--- trunk/jython/src/org/python/core/ParserFacade.java	2009-07-05 14:55:44 UTC (rev 6508)
+++ trunk/jython/src/org/python/core/ParserFacade.java	2009-07-05 18:07:53 UTC (rev 6509)
@@ -25,7 +25,8 @@
 import org.python.antlr.NoCloseReaderStream;
 import org.python.antlr.ParseException;
 import org.python.antlr.PythonLexer;
-import org.python.antlr.PythonPartial;
+import org.python.antlr.PythonPartialLexer;
+import org.python.antlr.PythonPartialParser;
 import org.python.antlr.PythonTokenSource;
 import org.python.antlr.PythonTree;
 import org.python.antlr.base.mod;
@@ -183,16 +184,15 @@
     }
 
     private static boolean validPartialSentence(BufferedReader bufreader, CompileMode kind, String filename) {
-        PythonLexer lexer = null;
+        PythonPartialLexer lexer = null;
         try {
             bufreader.reset();
             CharStream cs = new NoCloseReaderStream(bufreader);
-            lexer = new BaseParser.PyLexer(cs);
-            lexer.partial = true;
+            lexer = new BaseParser.PyPartialLexer(cs);
             CommonTokenStream tokens = new CommonTokenStream(lexer);
             PythonTokenSource indentedSource = new PythonTokenSource(tokens, filename);
             tokens = new CommonTokenStream(indentedSource);
-            PythonPartial parser = new PythonPartial(tokens);
+            PythonPartialParser parser = new PythonPartialParser(tokens);
             switch (kind) {
             case single:
                 parser.single_input();

Modified: trunk/jython/tests/java/org/python/antlr/PythonPartialTester.java
===================================================================
--- trunk/jython/tests/java/org/python/antlr/PythonPartialTester.java	2009-07-05 14:55:44 UTC (rev 6508)
+++ trunk/jython/tests/java/org/python/antlr/PythonPartialTester.java	2009-07-05 18:07:53 UTC (rev 6509)
@@ -14,12 +14,12 @@
         try {
             PythonTree result = null;
             CharStream input = new ANTLRFileStream(args[0]);
-            PythonLexer lexer = new BaseParser.PyLexer(input);
+            PythonPartialLexer lexer = new BaseParser.PyPartialLexer(input);
             CommonTokenStream tokens = new CommonTokenStream(lexer);
             //PythonTokenSource indentedSource = new PythonTokenSource(tokens);
             PythonTokenSource indentedSource = new PythonTokenSource(tokens, "<test>");
             tokens = new CommonTokenStream(indentedSource);
-            PythonPartial parser = new PythonPartial(tokens);
+            PythonPartialParser parser = new PythonPartialParser(tokens);
             parser.single_input();
             System.out.println("SUCCEED");
         } catch (ParseException e) {


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6508] trunk/jython/grammar/PythonPartial.g

[<fwi...@us...>]

Revision: 6508
          http://jython.svn.sourceforge.net/jython/?rev=6508&view=rev
Author:   fwierzbicki
Date:     2009-07-05 14:55:44 +0000 (Sun, 05 Jul 2009)

Log Message:
-----------
Last of the "make PythonPartial.g diffable from Python.g". Now on to actually
fixing some PythonPartial problems.

Modified Paths:
--------------
    trunk/jython/grammar/PythonPartial.g

Modified: trunk/jython/grammar/PythonPartial.g
===================================================================
--- trunk/jython/grammar/PythonPartial.g	2009-07-03 23:10:43 UTC (rev 6507)
+++ trunk/jython/grammar/PythonPartial.g	2009-07-05 14:55:44 UTC (rev 6508)
@@ -375,241 +375,562 @@
     | import_from
     ;
 
-import_name : IMPORT dotted_as_names
-            ;
+//import_name: 'import' dotted_as_names
+import_name
+    : IMPORT dotted_as_names
+   
+    ;
 
-import_from: FROM (DOT* dotted_name | DOT+) IMPORT
-              (STAR
-              | import_as_names
-              | LPAREN import_as_names RPAREN
-              )
-           ;
+//import_from: ('from' ('.'* dotted_name | '.'+)
+//              'import' ('*' | '(' import_as_names ')' | import_as_names))
+import_from
+    : FROM (DOT* dotted_name | DOT+) IMPORT 
+        (STAR
+       
+        | import_as_names
+       
+        | LPAREN import_as_names COMMA? RPAREN
+       
+        )
+    ;
 
-import_as_names : import_as_name (COMMA import_as_name)* (COMMA)?
-                ;
+//import_as_names: import_as_name (',' import_as_name)* [',']
+import_as_names 
+    : import_as_name (COMMA import_as_name)*
+    
+    ;
 
-import_as_name : NAME (AS NAME)?
-               ;
+//import_as_name: NAME [('as' | NAME) NAME]
+import_as_name 
 
-dotted_as_name : dotted_name (AS NAME)?
-               ;
+    : NAME (AS NAME)?
+    
+    ;
 
-dotted_as_names : dotted_as_name (COMMA dotted_as_name)*
-                ;
-dotted_name : NAME (DOT NAME)*
-            ;
+//XXX: when does CPython Grammar match "dotted_name NAME NAME"?
+//dotted_as_name: dotted_name [('as' | NAME) NAME]
+dotted_as_name 
 
-global_stmt : GLOBAL NAME (COMMA NAME)*
-            ;
 
-exec_stmt : EXEC expr (IN test (COMMA test)?)?
-          ;
+    : dotted_name (AS NAME)?
+    
+    ;
 
-assert_stmt : ASSERT test (COMMA test)?
-            ;
+//dotted_as_names: dotted_as_name (',' dotted_as_name)*
+dotted_as_names 
+    : dotted_as_name (COMMA dotted_as_name)*
+    
+    ;
 
-compound_stmt : if_stmt
-              | while_stmt
-              | for_stmt
-              | try_stmt
-              | with_stmt
-              | (decorators? DEF) => funcdef
-              | classdef
-              ;
+//dotted_name: NAME ('.' NAME)*
+dotted_name 
+    : NAME (DOT attr)* 
+    ;
 
-if_stmt: IF test COLON suite elif_clause*  (ORELSE COLON suite)?
-       ;
+//global_stmt: 'global' NAME (',' NAME)*
+global_stmt
+    : GLOBAL NAME (COMMA NAME)*
+   
+    ;
 
-elif_clause : ELIF test COLON suite
-            ;
+//exec_stmt: 'exec' expr ['in' test [',' test]]
+exec_stmt
 
-while_stmt : WHILE test COLON suite (ORELSE COLON suite)?
-           ;
+    : EXEC expr (IN test
+        (COMMA test)?)?
+      
+    ;
 
-for_stmt : FOR exprlist IN testlist COLON suite (ELSE COLON suite)?
-         ;
+//assert_stmt: 'assert' test [',' test]
+assert_stmt
+    : ASSERT test (COMMA test)?
+   
+    ;
 
-try_stmt : TRY COLON suite
-           ( except_clause+ (ELSE COLON suite)? (FINALLY COLON suite)?
-           | FINALLY COLON suite
-           )?
-         ;
+//compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | funcdef | classdef
+compound_stmt
+    : if_stmt
+    | while_stmt
+    | for_stmt
+    | try_stmt
+    | with_stmt
+    | (decorators? DEF) => funcdef
+    | classdef
+    ;
 
-with_stmt: WITH test (with_var)? COLON suite
-         ;
+//if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
+if_stmt
+    : IF test COLON suite elif_clause?
+   
+    ;
 
-with_var: (AS | NAME) expr
-        ;
+//not in CPython's Grammar file
+elif_clause  
+    : else_clause 
+    | ELIF test COLON suite
+        (elif_clause
+       
+        |
+       
+        )
+    ;
 
-except_clause : EXCEPT (test (COMMA test)?)? COLON suite
-              ;
+//not in CPython's Grammar file
+else_clause 
+    : ORELSE COLON suite 
+    ;
 
-suite : simple_stmt
-      | NEWLINE (EOF
-                | (DEDENT)+ EOF
-                |INDENT (stmt)+ (DEDENT
-                                |EOF
-                                )
-                )
+//while_stmt: 'while' test ':' suite ['else' ':' suite]
+while_stmt
+
+    : WHILE test COLON suite (ORELSE COLON suite)?
+    
+    ;
+
+//for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]
+for_stmt
+
+    : FOR exprlist IN testlist COLON suite
+        (ORELSE COLON suite)?
+      
+    ;
+
+//try_stmt: ('try' ':' suite
+//           ((except_clause ':' suite)+
+//           ['else' ':' suite]
+//           ['finally' ':' suite] |
+//           'finally' ':' suite))
+try_stmt
+
+    : TRY COLON suite
+      ( except_clause+ (ORELSE COLON suite)? (FINALLY COLON suite)?
+        
+      | FINALLY COLON suite
+        
+      )
       ;
 
-test: or_test
-    ( (IF or_test ELSE) => IF or_test ELSE test)?
+//with_stmt: 'with' test [ with_var ] ':' suite
+with_stmt
+
+    : WITH test (with_var)? COLON suite
+      
+    ;
+
+//with_var: ('as' | NAME) expr
+with_var 
+    : (AS | NAME) expr
+      
+    ;
+
+//except_clause: 'except' [test [',' test]]
+except_clause
+    : EXCEPT (test (COMMA test)?)? COLON suite
+   
+    ;
+
+//suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT
+suite 
+
+    : simple_stmt
+      
+    | NEWLINE (EOF
+              | (DEDENT)+ EOF
+              |INDENT (stmt)+ (DEDENT
+                              |EOF
+                              )
+              )
+    ;
+
+//test: or_test ['if' or_test 'else' test] | lambdef
+test
+    :or_test
+      ( (IF or_test ORELSE) => IF or_test ORELSE test
+     
+      |
+     
+      )
     | lambdef
     ;
 
-or_test : and_test (OR and_test)*
-        ;
+//or_test: and_test ('or' and_test)*
+or_test
+    : and_test
+        ( (OR and_test
+          )+
+        |
+       
+        )
+    ;
 
-and_test : not_test (AND not_test)*
-         ;
+//and_test: not_test ('and' not_test)*
+and_test
+    : not_test
+        ( (AND not_test
+          )+
+        |
+       
+        )
+    ;
 
-not_test : NOT not_test
-         | comparison
-         ;
+//not_test: 'not' not_test | comparison
+not_test
+    : NOT not_test
+   
+    | comparison
+    ;
 
-comparison: expr (comp_op expr)*
-          ;
+//comparison: expr (comp_op expr)*
+comparison
+    : expr
+       ( ( comp_op expr 
+         )+
+       |
+      
+       )
+    ;
 
-comp_op : LESS
-        | GREATER
-        | EQUAL
-        | GREATEREQUAL
-        | LESSEQUAL
-        | ALT_NOTEQUAL
-        | NOTEQUAL
-        | IN
-        | NOT IN
-        | IS
-        | IS NOT
-        ;
+//comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not'
+comp_op 
+    : LESS 
+    | GREATER 
+    | EQUAL 
+    | GREATEREQUAL 
+    | LESSEQUAL 
+    | ALT_NOTEQUAL 
+    | NOTEQUAL 
+    | IN 
+    | NOT IN 
+    | IS 
+    | IS NOT 
+    ;
 
-expr : xor_expr (VBAR xor_expr)*
-     ;
 
-xor_expr : and_expr (CIRCUMFLEX and_expr)*
-         ;
+//expr: xor_expr ('|' xor_expr)*
+expr
+    : xor_expr
+        ( (VBAR xor_expr
+          )+
+        |
+       
+        )
+    ;
 
-and_expr : shift_expr (AMPER shift_expr)*
-         ;
 
-shift_expr : arith_expr ((LEFTSHIFT|RIGHTSHIFT) arith_expr)*
-           ;
+//xor_expr: and_expr ('^' and_expr)*
+xor_expr
 
-arith_expr: term ((PLUS|MINUS) term)*
-          ;
+    : and_expr
+        ( (CIRCUMFLEX and_expr
+          )+
+        |
+       
+        )
+    ;
 
-term : factor ((STAR | SLASH | PERCENT | DOUBLESLASH ) factor)*
-     ;
+//and_expr: shift_expr ('&' shift_expr)*
+and_expr
 
-factor : PLUS factor
-       | MINUS factor
-       | TILDE factor
-       | power
-       | TRAILBACKSLASH
-       ;
+    : shift_expr
+        ( (AMPER shift_expr
+          )+
+        |
+       
+        )
+    ;
 
-power : atom (trailer)* (options {greedy=true;}:DOUBLESTAR factor)?
-      ;
+//shift_expr: arith_expr (('<<'|'>>') arith_expr)*
+shift_expr
 
-atom : LPAREN 
-       ( yield_expr
-       | testlist_gexp
-       )?
-       RPAREN
-     | LBRACK (listmaker)? RBRACK
-     | LCURLY (dictmaker)? RCURLY
+    : arith_expr
+        ( ( shift_op arith_expr 
+          )+
+        |
+       
+        )
+    ;
+
+shift_op 
+    : LEFTSHIFT 
+    | RIGHTSHIFT 
+    ;
+
+//arith_expr: term (('+'|'-') term)*
+arith_expr
+
+    : term
+        ( (arith_op term 
+          )+
+        |
+       
+        )
+    ;
+
+arith_op 
+    : PLUS 
+    | MINUS 
+    ;
+
+//term: factor (('*'|'/'|'%'|'//') factor)*
+term
+
+    : factor
+        ( (term_op factor 
+          )+
+        |
+       
+        )
+    ;
+
+term_op 
+    :STAR 
+    |SLASH 
+    |PERCENT 
+    |DOUBLESLASH 
+    ;
+
+//factor: ('+'|'-'|'~') factor | power
+factor 
+
+    : PLUS factor 
+    | MINUS factor 
+    | TILDE factor 
+    | power 
+    | TRAILBACKSLASH
+    ;
+
+//power: atom trailer* ['**' factor]
+power 
+
+    : atom (trailer)* (options {greedy=true;}:DOUBLESTAR factor)?
+      
+    ;
+
+//atom: ('(' [yield_expr|testlist_gexp] ')' |
+//       '[' [listmaker] ']' |
+//       '{' [dictmaker] '}' |
+//       '`' testlist1 '`' |
+//       NAME | NUMBER | STRING+)
+atom
+    : LPAREN 
+      ( yield_expr
+     
+      | testlist_gexp
+     
+      |
+     
+      )
+      RPAREN
+    | LBRACK
+      (listmaker
+     
+      |
+     
+      )
+      RBRACK
+    | LCURLY 
+       (dictmaker
+      
+       |
+      
+       )
+       RCURLY
      | BACKQUOTE testlist BACKQUOTE
+    
      | NAME
+    
      | INT
+    
      | LONGINT
+    
      | FLOAT
+    
      | COMPLEX
-     | (STRING)+
+    
+     | (STRING)+ 
+    
      | STRINGPART
      ;
 
-listmaker : test 
-            ( list_for
-            | (options {greedy=true;}:COMMA test)*
-            ) (COMMA)?
+//listmaker: test ( list_for | (',' test)* [','] )
+listmaker
+    : test 
+        (list_for
+          
+        | (options {greedy=true;}:COMMA test)*
+          
+        ) (COMMA)?
           ;
 
+//testlist_gexp: test ( gen_for | (',' test)* [','] )
 testlist_gexp
-    : test ( (options {k=2;}: COMMA test)* (COMMA)?
-           | gen_for
-           )
+
+    : test
+        ( ((options {k=2;}: COMMA test)* (COMMA)?
+         
+         
+          )
+        | (gen_for
            
+          )
+        )
     ;
 
-lambdef: LABMDA (varargslist)? COLON test
-       ;
+//lambdef: 'lambda' [varargslist] ':' test
+lambdef
 
-trailer : LPAREN (arglist)? RPAREN
-        | LBRACK subscriptlist RBRACK
-        | DOT NAME
-        ;
+    : LAMBDA (varargslist)? COLON test
+      
+    ;
 
-subscriptlist : subscript (options {greedy=true;}:COMMA subscript)* (COMMA)?
-              ;
+//trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME
+trailer 
+    : LPAREN 
+        (arglist
+       
+        |
+       
+        )
+      RPAREN
+    | LBRACK subscriptlist RBRACK
+   
+    | DOT attr
+   
+    ;
 
-subscript : DOT DOT DOT
-          | test (COLON (test)? (sliceop)?)?
-          | COLON (test)? (sliceop)?
-          ;
+//subscriptlist: subscript (',' subscript)* [',']
+subscriptlist
+    : subscript (options {greedy=true;}:COMMA subscript)* (COMMA)?
+      
+    ;
 
-sliceop : COLON (test)?
-        ;
+//subscript: '.' '.' '.' | test | [test] ':' [test] [sliceop]
+subscript 
 
-exprlist : expr (options {k=2;}: COMMA expr)* (COMMA)?
-         ;
+    : DOT DOT DOT
+   
+    | (test COLON)
+   => test (COLON (test)? (sliceop)?)?
+      
+    | (COLON)
+   => COLON (test)? (sliceop)?
+      
+    | test
+   
+    ;
 
+//sliceop: ':' [test]
+sliceop
+    : COLON
+     (test 
+     |
+     )
+    ;
+
+//exprlist: expr (',' expr)* [',']
+exprlist 
+    : (expr COMMA) => expr (options {k=2;}: COMMA expr)* (COMMA)?
+      
+    | expr
+      
+    ;
+
+//not in CPython's Grammar file
+//Needed as an exprlist that does not produce tuples for del_stmt.
+del_list 
+    : expr (options {k=2;}: COMMA expr)* (COMMA)?
+      
+    ;
+
+//testlist: test (',' test)* [',']
 testlist
-    : test (options {k=2;}: COMMA test)* (COMMA)?
+    : (test COMMA)
+   => test (options {k=2;}: COMMA test)* (COMMA)?
+   
+    | test
     ;
 
-dictmaker : test COLON test (options {k=2;}:COMMA test COLON test)* (COMMA)?
-          ;
+//dictmaker: test ':' test (',' test ':' test)* [',']
+dictmaker 
+    : test COLON test
+        (options {k=2;}:COMMA test COLON test)*
+        (COMMA)?
+      
+    ;
 
-classdef: decorators (CLASS NAME (LPAREN testlist? RPAREN)? COLON suite)?
-        | CLASS NAME (LPAREN testlist? RPAREN)? COLON suite
-        ;
+//classdef: 'class' NAME ['(' [testlist] ')'] ':' suite
+classdef
 
-arglist : argument (COMMA argument)*
-          ( COMMA
-            ( STAR test (COMMA DOUBLESTAR test)?
-            | DOUBLESTAR test
-            )?
+    : decorators? CLASS NAME (LPAREN testlist? RPAREN)? COLON suite
+      
+    ;
+
+//arglist: (argument ',')* (argument [',']| '*' test [',' '**' test] | '**' test)
+arglist 
+
+    : argument (COMMA argument)*
+          (COMMA
+              ( STAR test (COMMA DOUBLESTAR test)?
+              | DOUBLESTAR test
+              )?
           )?
-        |   STAR test (COMMA DOUBLESTAR test)?
-        |   DOUBLESTAR test
-        ;
+      
+    | STAR test (COMMA DOUBLESTAR test)?
+      
+    | DOUBLESTAR test
+      
+    ;
 
-argument : test ( (ASSIGN test) | gen_for)?
-         ;
+//argument: test [gen_for] | test '=' test  # Really [keyword '='] test
+argument 
+    : test
+        ((ASSIGN test)
+          
+        | gen_for
+          
+        | 
+        )
+    ;
 
-list_iter : list_for
-          | list_if
-          ;
+//list_iter: list_for | list_if
+list_iter 
+    : list_for
+    | list_if
+    ;
 
-list_for : FOR exprlist IN testlist (list_iter)?
-         ;
+//list_for: 'for' exprlist 'in' testlist_safe [list_iter]
+list_for 
+    : FOR exprlist IN testlist (list_iter)?
+      
+    ;
 
-list_if : IF test (list_iter)?
-        ;
+//list_if: 'if' test [list_iter]
+list_if
+    : IF test (list_iter)?
+    
+    ;
 
-gen_iter: gen_for
-        | gen_if
-        ;
+//gen_iter: gen_for | gen_if
+gen_iter 
+    : gen_for
+    | gen_if
+    ;
 
-gen_for: FOR exprlist IN or_test gen_iter?
-       ;
+//gen_for: 'for' exprlist 'in' or_test [gen_iter]
+gen_for 
+    : FOR exprlist IN or_test gen_iter?
+      
+    ;
 
-gen_if: IF test gen_iter?
-      ;
+//gen_if: 'if' old_test [gen_iter]
+gen_if
+    : IF test gen_iter?
+    
+    ;
 
-yield_expr : YIELD testlist?
-           ;
-//XXX:
-//testlist1: test (',' test)*
+//yield_expr: 'yield' [testlist]
+yield_expr
+    : YIELD testlist?
+   
+    ;
 


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6507] trunk/jython/grammar

[<fwi...@us...>]

Revision: 6507
          http://jython.svn.sourceforge.net/jython/?rev=6507&view=rev
Author:   fwierzbicki
Date:     2009-07-03 23:10:43 +0000 (Fri, 03 Jul 2009)

Log Message:
-----------
Make PythonPartial easier to diff w/ Python.g, and small whitespace fix in
Python.g.

Modified Paths:
--------------
    trunk/jython/grammar/Python.g
    trunk/jython/grammar/PythonPartial.g

Modified: trunk/jython/grammar/Python.g
===================================================================
--- trunk/jython/grammar/Python.g	2009-07-03 15:45:08 UTC (rev 6506)
+++ trunk/jython/grammar/Python.g	2009-07-03 23:10:43 UTC (rev 6507)
@@ -620,7 +620,7 @@
       |
      -> ^(PRINT<Print>[$PRINT, null, new ArrayList<expr>(), true])
       )
-           ;
+      ;
 
 //not in CPython's Grammar file
 printlist returns [boolean newline, List elts]

Modified: trunk/jython/grammar/PythonPartial.g
===================================================================
--- trunk/jython/grammar/PythonPartial.g	2009-07-03 15:45:08 UTC (rev 6506)
+++ trunk/jython/grammar/PythonPartial.g	2009-07-03 23:10:43 UTC (rev 6507)
@@ -255,78 +255,125 @@
            | assert_stmt
            ;
 
-expr_stmt : testlist
-            ( augassign yield_expr
-            | augassign testlist
-            | assigns
-            )?
-          ;
+expr_stmt 
 
-assigns
-    : assign_testlist+
-    | assign_yield+
+    : ((testlist augassign) => testlist
+        ( (augassign yield_expr 
+            
+          )
+        | (augassign testlist
+            
+          )
+        )
+    | (testlist ASSIGN) => testlist
+        (
+        | ((ASSIGN testlist)+
+       
+          )
+        | ((ASSIGN yield_expr)+
+       
+          )
+        )
+    | testlist
+      
+    )
     ;
 
-assign_testlist
-       : ASSIGN testlist
-       ;
-
-assign_yield
-    : ASSIGN yield_expr
+//augassign: ('+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' |
+//            '<<=' | '>>=' | '**=' | '//=')
+augassign 
+    : PLUSEQUAL 
+    | MINUSEQUAL 
+    | STAREQUAL 
+    | SLASHEQUAL 
+    | PERCENTEQUAL 
+    | AMPEREQUAL 
+    | VBAREQUAL 
+    | CIRCUMFLEXEQUAL 
+    | LEFTSHIFTEQUAL 
+    | RIGHTSHIFTEQUAL 
+    | DOUBLESTAREQUAL 
+    | DOUBLESLASHEQUAL 
     ;
 
-augassign : PLUSEQUAL
-          | MINUSEQUAL
-          | STAREQUAL
-          | SLASHEQUAL
-          | PERCENTEQUAL
-          | AMPEREQUAL
-          | VBAREQUAL
-          | CIRCUMFLEXEQUAL
-          | LEFTSHIFTEQUAL
-          | RIGHTSHIFTEQUAL
-          | DOUBLESTAREQUAL
-          | DOUBLESLASHEQUAL
-          ;
+//print_stmt: 'print' ( [ test (',' test)* [','] ] |
+//                      '>>' test [ (',' test)+ [','] ] )
+print_stmt
+    : PRINT 
+      (printlist
+     
+      | RIGHTSHIFT printlist
+     
+      |
+     
+      )
+      ;
 
-print_stmt : PRINT (printlist | RIGHTSHIFT printlist)?
-           ;
-
+//not in CPython's Grammar file
 printlist returns [boolean newline]
     : test (options {k=2;}: COMMA test)* (COMMA)?
     ;
 
-del_stmt : DELETE exprlist
-         ;
+//del_stmt: 'del' exprlist
+del_stmt
+    : DELETE exprlist
+   
+    ;
 
-pass_stmt : PASS
-          ;
+//pass_stmt: 'pass'
+pass_stmt
+    : PASS
+   
+    ;
 
-flow_stmt : break_stmt
-          | continue_stmt
-          | return_stmt
-          | raise_stmt
-          | yield_stmt
-          ;
+//flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt
+flow_stmt
+    : break_stmt
+    | continue_stmt
+    | return_stmt
+    | raise_stmt
+    | yield_stmt
+    ;
 
-break_stmt : BREAK
-           ;
+//break_stmt: 'break'
+break_stmt
+    : BREAK
+   
+    ;
 
-continue_stmt : CONTINUE
-              ;
+//continue_stmt: 'continue'
+continue_stmt
+    : CONTINUE 
+   
+    ;
 
-return_stmt : RETURN (testlist)?
-            ;
+//return_stmt: 'return' [testlist]
+return_stmt
+    : RETURN 
+      (testlist
+     
+      |
+     
+      )
+      ;
 
-yield_stmt : yield_expr
-           ;
+//yield_stmt: yield_expr
+yield_stmt
+    : yield_expr 
+    ;
 
-raise_stmt: RAISE (test (COMMA test (COMMA test)?)?)?
-          ;
+//raise_stmt: 'raise' [test [',' test [',' test]]]
+raise_stmt
+    : RAISE (test (COMMA test
+        (COMMA test)?)?)?
+   
+    ;
 
-import_stmt : import_name
-            | import_from
-            ;
+//import_stmt: import_name | import_from
+import_stmt
+    : import_name
+    | import_from
+    ;
 
 import_name : IMPORT dotted_as_names
             ;


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6506] trunk/jython/Lib

[<am...@us...>]

Revision: 6506
          http://jython.svn.sourceforge.net/jython/?rev=6506&view=rev
Author:   amak
Date:     2009-07-03 15:45:08 +0000 (Fri, 03 Jul 2009)

Log Message:
-----------
1. Added support for the AI_PASSIVE flag
2. Added support for the AI_CANONNAME flag
3. Added unit tests for same
4. Split out getaddrinfo unit tests into their own test case class

Modified Paths:
--------------
    trunk/jython/Lib/socket.py
    trunk/jython/Lib/test/test_socket.py

Modified: trunk/jython/Lib/socket.py
===================================================================
--- trunk/jython/Lib/socket.py	2009-06-30 06:11:20 UTC (rev 6505)
+++ trunk/jython/Lib/socket.py	2009-07-03 15:45:08 UTC (rev 6506)
@@ -76,12 +76,12 @@
 
 # Javax.net.ssl classes
 import javax.net.ssl.SSLSocketFactory
-# Javax.net.ssl exceptions
-javax.net.ssl.SSLException
-javax.net.ssl.SSLHandshakeException
-javax.net.ssl.SSLKeyException
-javax.net.ssl.SSLPeerUnverifiedException
-javax.net.ssl.SSLProtocolException
+# Javax.net.ssl exceptions
+javax.net.ssl.SSLException
+javax.net.ssl.SSLHandshakeException
+javax.net.ssl.SSLKeyException
+javax.net.ssl.SSLPeerUnverifiedException
+javax.net.ssl.SSLProtocolException
 
 import org.python.core.io.DatagramSocketIO
 import org.python.core.io.ServerSocketIO
@@ -128,16 +128,16 @@
 (java.nio.channels.NotYetConnectedException, ALL)        : None,
 (java.nio.channels.UnresolvedAddressException, ALL)      : lambda: gaierror(errno.EGETADDRINFOFAILED, 'getaddrinfo failed'),
 (java.nio.channels.UnsupportedAddressTypeException, ALL) : None,
-
-# These error codes are currently wrong: getting them correct is going to require
-# some investigation. Cpython 2.6 introduced extensive SSL support.
-
-(javax.net.ssl.SSLException, ALL)                        : lambda: sslerror(-1, 'SSL exception'),
-(javax.net.ssl.SSLHandshakeException, ALL)               : lambda: sslerror(-1, 'SSL handshake exception'),
-(javax.net.ssl.SSLKeyException, ALL)                     : lambda: sslerror(-1, 'SSL key exception'),
-(javax.net.ssl.SSLPeerUnverifiedException, ALL)          : lambda: sslerror(-1, 'SSL peer unverified exception'),
-(javax.net.ssl.SSLProtocolException, ALL)                : lambda: sslerror(-1, 'SSL protocol exception'),
 
+# These error codes are currently wrong: getting them correct is going to require
+# some investigation. Cpython 2.6 introduced extensive SSL support.
+
+(javax.net.ssl.SSLException, ALL)                        : lambda: sslerror(-1, 'SSL exception'),
+(javax.net.ssl.SSLHandshakeException, ALL)               : lambda: sslerror(-1, 'SSL handshake exception'),
+(javax.net.ssl.SSLKeyException, ALL)                     : lambda: sslerror(-1, 'SSL key exception'),
+(javax.net.ssl.SSLPeerUnverifiedException, ALL)          : lambda: sslerror(-1, 'SSL peer unverified exception'),
+(javax.net.ssl.SSLProtocolException, ALL)                : lambda: sslerror(-1, 'SSL protocol exception'),
+
 }
 
 def would_block_error(exc=None):
@@ -168,7 +168,14 @@
 AF_INET6 = 23
 
 AI_PASSIVE=1
+AI_CANONNAME=2
 
+# For some reason, probably historical, SOCK_DGRAM and SOCK_STREAM are opposite values of what they are on cpython.
+# I.E. The following is the way they are on cpython
+# SOCK_STREAM    = 1
+# SOCK_DGRAM     = 2
+# At some point, we should probably switch them around, which *should* not affect anybody
+
 SOCK_DGRAM     = 1
 SOCK_STREAM    = 2
 SOCK_RAW       = 3 # not supported
@@ -189,10 +196,10 @@
 SO_TIMEOUT     = 128
 
 TCP_NODELAY    = 256
-
-INADDR_ANY = "0.0.0.0"
-INADDR_BROADCAST = "255.255.255.255"
 
+INADDR_ANY = "0.0.0.0"
+INADDR_BROADCAST = "255.255.255.255"
+
 # Options with negative constants are not supported
 # They are being added here so that code that refers to them
 # will not break with an AttributeError
@@ -209,11 +216,11 @@
 SO_SNDTIMEO         = -512
 SO_TYPE             = -1024
 SO_USELOOPBACK      = -2048
-
+
 __all__ = ['AF_UNSPEC', 'AF_INET', 'AF_INET6', 'AI_PASSIVE', 'SOCK_DGRAM',
         'SOCK_RAW', 'SOCK_RDM', 'SOCK_SEQPACKET', 'SOCK_STREAM', 'SOL_SOCKET',
         'SO_BROADCAST', 'SO_ERROR', 'SO_KEEPALIVE', 'SO_LINGER', 'SO_OOBINLINE',
-        'SO_RCVBUF', 'SO_REUSEADDR', 'SO_SNDBUF', 'SO_TIMEOUT', 'TCP_NODELAY',
+        'SO_RCVBUF', 'SO_REUSEADDR', 'SO_SNDBUF', 'SO_TIMEOUT', 'TCP_NODELAY',
         'INADDR_ANY', 'INADDR_BROADCAST', 'IPPROTO_TCP', 'IPPROTO_UDP', 
         'SocketType', 'error', 'herror', 'gaierror', 'timeout',
         'getfqdn', 'gethostbyaddr', 'gethostbyname', 'gethostname',
@@ -222,15 +229,15 @@
         'SHUT_RD', 'SHUT_WR', 'SHUT_RDWR',
         ]
 
-def _constant_to_name(const_value):
-    sock_module = sys.modules['socket']
-    try:
-        for name in dir(sock_module):
-            if getattr(sock_module, name) is const_value:
-                return name
-        return "Unknown"
-    finally:
-        sock_module = None
+def _constant_to_name(const_value):
+    sock_module = sys.modules['socket']
+    try:
+        for name in dir(sock_module):
+            if getattr(sock_module, name) is const_value:
+                return name
+        return "Unknown"
+    finally:
+        sock_module = None
 
 class _nio_impl:
 
@@ -285,15 +292,15 @@
 
 class _client_socket_impl(_nio_impl):
 
-    options = {
+    options = {
         (SOL_SOCKET,  SO_KEEPALIVE):   'KeepAlive',
         (SOL_SOCKET,  SO_LINGER):      'SoLinger',
         (SOL_SOCKET,  SO_OOBINLINE):   'OOBInline',
         (SOL_SOCKET,  SO_RCVBUF):      'ReceiveBufferSize',
         (SOL_SOCKET,  SO_REUSEADDR):   'ReuseAddress',
         (SOL_SOCKET,  SO_SNDBUF):      'SendBufferSize',
-        (SOL_SOCKET,  SO_TIMEOUT):     'SoTimeout',
-        (IPPROTO_TCP, TCP_NODELAY):    'TcpNoDelay',
+        (SOL_SOCKET,  SO_TIMEOUT):     'SoTimeout',
+        (IPPROTO_TCP, TCP_NODELAY):    'TcpNoDelay',
     }
 
     def __init__(self, socket=None):
@@ -361,10 +368,10 @@
 
 class _server_socket_impl(_nio_impl):
 
-    options = {
+    options = {
         (SOL_SOCKET, SO_RCVBUF):      'ReceiveBufferSize',
         (SOL_SOCKET, SO_REUSEADDR):   'ReuseAddress',
-        (SOL_SOCKET, SO_TIMEOUT):     'SoTimeout',
+        (SOL_SOCKET, SO_TIMEOUT):     'SoTimeout',
     }
 
     def __init__(self, host, port, backlog, reuse_addr):
@@ -399,12 +406,12 @@
 
 class _datagram_socket_impl(_nio_impl):
 
-    options = {
+    options = {
         (SOL_SOCKET, SO_BROADCAST):   'Broadcast',
         (SOL_SOCKET, SO_RCVBUF):      'ReceiveBufferSize',
         (SOL_SOCKET, SO_REUSEADDR):   'ReuseAddress',
         (SOL_SOCKET, SO_SNDBUF):      'SendBufferSize',
-        (SOL_SOCKET, SO_TIMEOUT):     'SoTimeout',
+        (SOL_SOCKET, SO_TIMEOUT):     'SoTimeout',
     }
 
     def __init__(self, port=None, address=None, reuse_addr=0):
@@ -515,7 +522,7 @@
 has_ipv6 = False
 
 # Name and address functions
-
+
 def _gethostbyaddr(name):
     # This is as close as I can get; at least the types are correct...
     addresses = java.net.InetAddress.getAllByName(gethostbyname(name))
@@ -576,12 +583,12 @@
     assert family == AF_INET, "Only AF_INET sockets are currently supported on jython"
     assert type in (SOCK_DGRAM, SOCK_STREAM), "Only SOCK_STREAM and SOCK_DGRAM sockets are currently supported on jython"
     if type == SOCK_STREAM:
-        if protocol != 0:
-            assert protocol == IPPROTO_TCP, "Only IPPROTO_TCP supported on SOCK_STREAM sockets"
+        if protocol != 0:
+            assert protocol == IPPROTO_TCP, "Only IPPROTO_TCP supported on SOCK_STREAM sockets"
         return _tcpsocket()
     else:
-        if protocol != 0:
-            assert protocol == IPPROTO_UDP, "Only IPPROTO_UDP supported on SOCK_DGRAM sockets"
+        if protocol != 0:
+            assert protocol == IPPROTO_UDP, "Only IPPROTO_UDP supported on SOCK_DGRAM sockets"
         return _udpsocket()
 
 def getaddrinfo(host, port, family=AF_INET, socktype=None, proto=0, flags=None):
@@ -594,16 +601,23 @@
             AF_INET6:  lambda x: isinstance(x, java.net.Inet6Address),
             AF_UNSPEC: lambda x: isinstance(x, java.net.InetAddress),
         }[family])
-        # Cant see a way to support AI_PASSIVE right now.
-        # if flags and flags & AI_PASSIVE:
-        #     pass
+        if host == "":
+            host = java.net.InetAddress.getLocalHost().getHostName()
+        passive_mode = flags is not None and flags & AI_PASSIVE
+        canonname_mode = flags is not None and flags & AI_CANONNAME
         results = []
         for a in java.net.InetAddress.getAllByName(host):
             if len([f for f in filter_fns if f(a)]):
                 family = {java.net.Inet4Address: AF_INET, java.net.Inet6Address: AF_INET6}[a.getClass()]
+                if passive_mode and not canonname_mode:
+                    canonname = ""
+                else:
+                    canonname = asPyString(a.getCanonicalHostName())
+                if host is None and passive_mode and not canonname_mode:
+                    sockname = INADDR_ANY
+                else:
+                    sockname = asPyString(a.getHostAddress())
                 # TODO: Include flowinfo and scopeid in a 4-tuple for IPv6 addresses
-                canonname = asPyString(a.getCanonicalHostName())
-                sockname = asPyString(a.getHostAddress())
                 results.append((family, socktype, proto, canonname, (sockname, port)))
         return results
     except java.lang.Exception, jlx:
@@ -674,9 +688,9 @@
     close_lock = threading.Lock()
 
     def __init__(self):
-        self.timeout = _defaulttimeout
-        if self.timeout is not None:
-            self.mode = MODE_TIMEOUT
+        self.timeout = _defaulttimeout
+        if self.timeout is not None:
+            self.mode = MODE_TIMEOUT
         self.pending_options = {
             (SOL_SOCKET, SO_REUSEADDR):  0,
         }
@@ -844,7 +858,7 @@
                 self.sock_impl.bind(bind_host, bind_port, self.pending_options[ (SOL_SOCKET, SO_REUSEADDR) ])
             self._config() # Configure timeouts, etc, now that the socket exists
             self.sock_impl.connect(host, port)
-        except java.lang.Exception, jlx:
+        except java.lang.Exception, jlx:
             raise _map_exception(jlx)
 
     def connect(self, addr):
@@ -952,8 +966,8 @@
     def bind(self, addr):
         try:
             assert not self.sock_impl
-            host, port = _unpack_address_tuple(addr)
-            if host == "":
+            host, port = _unpack_address_tuple(addr)
+            if host == "":
                 host = INADDR_ANY
             host_address = java.net.InetAddress.getByName(host)
             self.sock_impl = _datagram_socket_impl(port, host_address, self.pending_options[ (SOL_SOCKET, SO_REUSEADDR) ])
@@ -994,8 +1008,8 @@
             if not self.sock_impl:
                 self.sock_impl = _datagram_socket_impl()
                 self._config()
-            host, port = _unpack_address_tuple(addr)
-            if host == "<broadcast>":
+            host, port = _unpack_address_tuple(addr)
+            if host == "<broadcast>":
                 host = INADDR_BROADCAST
             byte_array = java.lang.String(data).getBytes('iso-8859-1')
             result = self.sock_impl.sendto(byte_array, host, port, flags)
@@ -1409,11 +1423,11 @@
 
 class ssl:
 
-    def __init__(self, plain_sock, keyfile=None, certfile=None):
+    def __init__(self, plain_sock, keyfile=None, certfile=None):
         try:
             self.ssl_sock = self._make_ssl_socket(plain_sock)
             self._in_buf = java.io.BufferedInputStream(self.ssl_sock.getInputStream())
-            self._out_buf = java.io.BufferedOutputStream(self.ssl_sock.getOutputStream())
+            self._out_buf = java.io.BufferedOutputStream(self.ssl_sock.getOutputStream())
         except java.lang.Exception, jlx:
             raise _map_exception(jlx)
 
@@ -1429,7 +1443,7 @@
         return ssl_socket
 
     def read(self, n=4096):
-        try:
+        try:
             data = jarray.zeros(n, 'b')
             m = self._in_buf.read(data, 0, n)
             if m <= 0:
@@ -1441,7 +1455,7 @@
             raise _map_exception(jlx)
 
     def write(self, s):
-        try:
+        try:
             self._out_buf.write(s)
             self._out_buf.flush()
             return len(s)
@@ -1449,7 +1463,7 @@
             raise _map_exception(jlx)
 
     def _get_server_cert(self):
-        try:
+        try:
             return self.ssl_sock.getSession().getPeerCertificates()[0]
         except java.lang.Exception, jlx:
             raise _map_exception(jlx)

Modified: trunk/jython/Lib/test/test_socket.py
===================================================================
--- trunk/jython/Lib/test/test_socket.py	2009-06-30 06:11:20 UTC (rev 6505)
+++ trunk/jython/Lib/test/test_socket.py	2009-07-03 15:45:08 UTC (rev 6506)
@@ -2,6 +2,8 @@
 AMAK: 20050515: This module is the test_socket.py from cpython 2.4, ported to jython.
 """
 
+import java
+
 import unittest
 from test import test_support
 
@@ -308,22 +310,6 @@
         if not fqhn in all_host_names:
             self.fail("Error testing host resolution mechanisms.")
 
-    def testGetAddrInfo(self):
-        try:
-            socket.getaddrinfo(HOST, PORT, 9999)
-        except socket.gaierror, gaix:
-            self.failUnlessEqual(gaix[0], errno.EIO)
-        except Exception, x:
-            self.fail("getaddrinfo with bad family raised wrong exception: %s" % x)
-        else:
-            self.fail("getaddrinfo with bad family should have raised exception")
-            
-        addrinfos = socket.getaddrinfo(HOST, PORT)
-        for addrinfo in addrinfos:
-            family, socktype, proto, canonname, sockaddr = addrinfo
-            self.assert_(isinstance(canonname, str))
-            self.assert_(isinstance(sockaddr[0], str))
-
     def testRefCountGetNameInfo(self):
         # Testing reference count for getnameinfo
         import sys
@@ -529,7 +515,7 @@
 
 class TestSocketOptions(unittest.TestCase):
 
-    def setUp(self):
+    def setUp(self):
         self.test_udp = self.test_tcp_client = self.test_tcp_server = 0
 
     def _testSetAndGetOption(self, sock, level, option, values):
@@ -621,7 +607,7 @@
 
 class TestSupportedOptions(TestSocketOptions):
 
-    def testSO_BROADCAST(self):
+    def testSO_BROADCAST(self):
         self.test_udp = 1
         self._testOption(socket.SOL_SOCKET, socket.SO_BROADCAST, [0, 1])
 
@@ -821,32 +807,32 @@
     def _testDup(self):
         self.serv_conn.send(MSG)
         self.serv_conn.send('and ' + MSG)
-
-class UDPBindTest(unittest.TestCase):
-
+
+class UDPBindTest(unittest.TestCase):
+
     HOST = HOST
     PORT = PORT
 
-    def setUp(self):
+    def setUp(self):
         self.sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
-
+
     def testBindSpecific(self):
         self.sock.bind( (self.HOST, self.PORT) ) # Use a specific port
         actual_port = self.sock.getsockname()[1]
         self.failUnless(actual_port == self.PORT, 
             "Binding to specific port number should have returned same number: %d != %d" % (actual_port, self.PORT)) 
 
-    def testBindEphemeral(self):
+    def testBindEphemeral(self):
         self.sock.bind( (self.HOST, 0) ) # let system choose a free port
-        self.failUnless(self.sock.getsockname()[1] != 0, "Binding to port zero should have allocated an ephemeral port number") 
+        self.failUnless(self.sock.getsockname()[1] != 0, "Binding to port zero should have allocated an ephemeral port number") 
 
     def testShutdown(self):
         self.sock.bind( (self.HOST, self.PORT) )
         self.sock.shutdown(socket.SHUT_RDWR)
 
-    def tearDown(self):
-        self.sock.close()
-
+    def tearDown(self):
+        self.sock.close()
+
 class BasicUDPTest(ThreadedUDPSocketTest):
 
     def __init__(self, methodName='runTest'):
@@ -1354,7 +1340,7 @@
 used, but if it is on your network this failure is bogus.''' % host)
 
     def testConnectDefaultTimeout(self):
-        _saved_timeout = socket.getdefaulttimeout()
+        _saved_timeout = socket.getdefaulttimeout()
         socket.setdefaulttimeout(0.1)
         cli = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         host = '192.168.192.168'
@@ -1437,6 +1423,49 @@
         if not ok:
             self.fail("recv() returned success when we did not expect it")
 
+class TestGetAddrInfo(unittest.TestCase):
+
+    def testBadFamily(self):
+        try:
+            socket.getaddrinfo(HOST, PORT, 9999)
+        except socket.gaierror, gaix:
+            self.failUnlessEqual(gaix[0], errno.EIO)
+        except Exception, x:
+            self.fail("getaddrinfo with bad family raised wrong exception: %s" % x)
+        else:
+            self.fail("getaddrinfo with bad family should have raised exception")
+
+    def testReturnsAreStrings(self):
+        addrinfos = socket.getaddrinfo(HOST, PORT)
+        for addrinfo in addrinfos:
+            family, socktype, proto, canonname, sockaddr = addrinfo
+            self.assert_(isinstance(canonname, str))
+            self.assert_(isinstance(sockaddr[0], str))
+
+    def testAI_PASSIVE(self):
+        IPV4_LOOPBACK = "127.0.0.1"
+        local_hostname = java.net.InetAddress.getLocalHost().getHostName()
+        local_ip_address = java.net.InetAddress.getLocalHost().getHostAddress()
+        for flags, host_param, expected_canonname, expected_sockaddr in [
+            # First passive flag
+        	(socket.AI_PASSIVE, None, "", socket.INADDR_ANY),
+        	(socket.AI_PASSIVE, "", "", local_ip_address),
+        	(socket.AI_PASSIVE, "localhost", "", IPV4_LOOPBACK),
+        	(socket.AI_PASSIVE, local_hostname, "", local_ip_address),
+        	# Now passive flag AND canonname flag
+        	(socket.AI_PASSIVE|socket.AI_CANONNAME, None, "127.0.0.1", "127.0.0.1"),
+        	(socket.AI_PASSIVE|socket.AI_CANONNAME, "", local_hostname, local_ip_address),
+        	# The following result seems peculiar to me, and may be to do with my local machine setup
+        	# Also may be caused by a security permission failure.
+        	# If you have problems with the following result, just comment it out.
+        	(socket.AI_PASSIVE|socket.AI_CANONNAME, "localhost", IPV4_LOOPBACK, IPV4_LOOPBACK),
+        	(socket.AI_PASSIVE|socket.AI_CANONNAME, local_hostname, local_hostname, local_ip_address),
+        ]:
+            addrinfos = socket.getaddrinfo(host_param, 0, socket.AF_INET, socket.SOCK_STREAM, 0, flags)
+            for family, socktype, proto, canonname, sockaddr in addrinfos:
+                self.failUnlessEqual(expected_canonname, canonname, "For hostname '%s' and flags %d, canonname '%s' != '%s'" % (host_param, flags, expected_canonname, canonname) )
+                self.failUnlessEqual(expected_sockaddr, sockaddr[0], "For hostname '%s' and flags %d, sockaddr '%s' != '%s'" % (host_param, flags, expected_sockaddr, sockaddr[0]) )
+
 class TestExceptions(unittest.TestCase):
 
     def testExceptionTree(self):
@@ -1640,6 +1669,7 @@
         TCPClientTimeoutTest,
         TestExceptions,
         TestInvalidUsage,
+        TestGetAddrInfo,
         TestTCPAddressParameters,
         TestUDPAddressParameters,
         UDPBindTest,


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6505] trunk/jython/NEWS

[<cg...@us...>]

Revision: 6505
          http://jython.svn.sourceforge.net/jython/?rev=6505&view=rev
Author:   cgroves
Date:     2009-06-30 06:11:20 +0000 (Tue, 30 Jun 2009)

Log Message:
-----------
Keep forgetting to update this

Modified Paths:
--------------
    trunk/jython/NEWS

Modified: trunk/jython/NEWS
===================================================================
--- trunk/jython/NEWS	2009-06-30 06:09:12 UTC (rev 6504)
+++ trunk/jython/NEWS	2009-06-30 06:11:20 UTC (rev 6505)
@@ -6,6 +6,7 @@
     - [ 1859477 ] Dynamically loaded ServletFilters like PyServlet
   Bugs Fixed
     - [ 1377 ] Event names shadowed by a field name on Java types leads to a NPE
+    - [ 1381 ] Redundant declarations of interface implementation hides overriden methods
 
 Jython 2.5.0
   The same as rc4.


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6504] trunk/jython

[<cg...@us...>]

Revision: 6504
          http://jython.svn.sourceforge.net/jython/?rev=6504&view=rev
Author:   cgroves
Date:     2009-06-30 06:09:12 +0000 (Tue, 30 Jun 2009)

Log Message:
-----------
Add a test from Sven Reimers that actually tickles issue 1381 and fix it.

Modified Paths:
--------------
    trunk/jython/Lib/test/test_java_visibility.py
    trunk/jython/src/org/python/core/PyJavaType.java

Added Paths:
-----------
    trunk/jython/tests/java/org/python/tests/RedundantInterfaceDeclarations.java

Removed Paths:
-------------
    trunk/jython/tests/java/org/python/tests/DisagreeingInterfaceOverrides.java

Modified: trunk/jython/Lib/test/test_java_visibility.py
===================================================================
--- trunk/jython/Lib/test/test_java_visibility.py	2009-06-29 06:25:28 UTC (rev 6503)
+++ trunk/jython/Lib/test/test_java_visibility.py	2009-06-30 06:09:12 UTC (rev 6504)
@@ -5,10 +5,11 @@
 from test import test_support
 from java.lang import Byte, Class, Integer
 from java.util import ArrayList, Collections, HashMap, LinkedList, Observable, Observer 
-from org.python.tests import (Coercions, DisagreeingInterfaceOverrides, HiddenSuper,
-        InterfaceCombination, Invisible, Matryoshka, OnlySubclassable, OtherSubVisible,
-        SomePyMethods, SubVisible, Visible, VisibleOverride)
+from org.python.tests import (Coercions, HiddenSuper, InterfaceCombination, Invisible, Matryoshka,
+        OnlySubclassable, OtherSubVisible, SomePyMethods, SubVisible, Visible, VisibleOverride)
 from org.python.tests import VisibilityResults as Results
+from org.python.tests.RedundantInterfaceDeclarations import (Implementation, ExtraClass,
+        ExtraString, ExtraStringAndClass, ExtraClassAndString)
 
 class VisibilityTest(unittest.TestCase):
     def test_invisible(self):
@@ -157,14 +158,14 @@
         self.assertEquals("a string", synchList.remove(0))
 
     def test_interface_methods_merged(self):
-        '''Checks that implementing interfaces that use the same method name are all reachable.
+        '''Checks that declaring an interface redundantly doesn't hide merged methods.
 
         Bug #1381'''
-        imp = DisagreeingInterfaceOverrides.Implementation()
-        self.assertEquals("String", imp.call("string argument"))
-        self.assertEquals("int", imp.call(Integer(7)))
-        self.assertEquals("List", imp.call(LinkedList()))
-        self.assertEquals("ArrayList", imp.call(ArrayList()))
+        for impl in Implementation, ExtraString, ExtraClass, ExtraStringAndClass, ExtraClassAndString:
+            instance = impl()
+            self.assertEquals("String", instance.call("string argument"))
+            self.assertEquals("int", instance.call(7))
+            self.assertEquals("Class", instance.call(LinkedList))
 
 class JavaClassTest(unittest.TestCase):
     def test_class_methods_visible(self):

Modified: trunk/jython/src/org/python/core/PyJavaType.java
===================================================================
--- trunk/jython/src/org/python/core/PyJavaType.java	2009-06-29 06:25:28 UTC (rev 6503)
+++ trunk/jython/src/org/python/core/PyJavaType.java	2009-06-30 06:09:12 UTC (rev 6504)
@@ -199,6 +199,12 @@
                     // mro
                     continue;
                 }
+                if (baseClass != null && iface.isAssignableFrom(baseClass)) {
+                    // Don't include redundant interfaces. If the redundant interface has methods
+                    // that were combined with methods of the same name from other interfaces higher
+                    // in the hierarchy, adding it here hides the forms from those interfaces.
+                    continue;
+                }
                 visibleBases.add(PyType.fromClassSkippingInners(iface, needsInners));
             }
             if (javaProxy == Object.class) {

Deleted: trunk/jython/tests/java/org/python/tests/DisagreeingInterfaceOverrides.java
===================================================================
--- trunk/jython/tests/java/org/python/tests/DisagreeingInterfaceOverrides.java	2009-06-29 06:25:28 UTC (rev 6503)
+++ trunk/jython/tests/java/org/python/tests/DisagreeingInterfaceOverrides.java	2009-06-30 06:09:12 UTC (rev 6504)
@@ -1,51 +0,0 @@
-package org.python.tests;
-
-import java.util.ArrayList;
-import java.util.List;
-
-/**
- * Part of a test for issue #1381. It checks that Jython finds the proper overridden method when
- * dealing with several interfaces. The test itself is in
- * Lib/test_java_visibility.py#test_interface_methods_merged
- */
-public class DisagreeingInterfaceOverrides {
-
-    public interface StringArg {
-
-        String call(String arg);
-    }
-
-    public interface IntArg {
-
-        String call(int arg);
-    }
-
-    public interface ListArg {
-
-        String call(List<Object> arg);
-    }
-
-    public interface ArrayListArg {
-
-        String call(List<Object> arg);
-    }
-
-    public static class Implementation implements StringArg, IntArg, ListArg, ArrayListArg {
-
-        public String call(String arg) {
-            return "String";
-        }
-
-        public String call(int arg) {
-            return "int";
-        }
-
-        public String call(List<Object> arg) {
-            return "List";
-        }
-
-        public String call(ArrayList<Object> arg) {
-            return "ArrayList";
-        }
-    }
-}

Copied: trunk/jython/tests/java/org/python/tests/RedundantInterfaceDeclarations.java (from rev 6503, trunk/jython/tests/java/org/python/tests/DisagreeingInterfaceOverrides.java)
===================================================================
--- trunk/jython/tests/java/org/python/tests/RedundantInterfaceDeclarations.java	                        (rev 0)
+++ trunk/jython/tests/java/org/python/tests/RedundantInterfaceDeclarations.java	2009-06-30 06:09:12 UTC (rev 6504)
@@ -0,0 +1,49 @@
+package org.python.tests;
+
+/**
+ * Part of a test for issue #1381. It checks that Jython finds the proper implementation of an
+ * interface method when dealing with classes that have redundant declarations of implementing
+ * interfaces. The test itself is in Lib/test_java_visibility.py#test_interface_methods_merged
+ */
+public class RedundantInterfaceDeclarations {
+
+    public interface IntArg {
+
+        String call(int arg);
+    }
+
+    public interface ClassArg {
+
+        String call(Class<?> arg);
+    }
+
+    public interface StringArg extends ClassArg {
+
+        String call(String name);
+    }
+
+    public static abstract class AbstractImplementation implements StringArg, IntArg {
+
+        public String call(Class<?> arg) {
+            return "Class";
+        }
+    }
+
+    public static class Implementation extends AbstractImplementation implements StringArg {
+        public String call(String name) {
+            return "String";
+        }
+
+        public String call(int arg) {
+            return "int";
+        }
+    }
+
+    public static class ExtraString extends Implementation implements StringArg {}
+
+    public static class ExtraClass extends Implementation implements ClassArg {}
+
+    public static class ExtraStringAndClass extends Implementation implements StringArg, ClassArg {}
+
+    public static class ExtraClassAndString extends Implementation implements ClassArg, StringArg {}
+}


Property changes on: trunk/jython/tests/java/org/python/tests/RedundantInterfaceDeclarations.java
___________________________________________________________________
Added: svn:mergeinfo
   + 


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6503] trunk/jython/src/org/python/util

[<cg...@us...>]

Revision: 6503
          http://jython.svn.sourceforge.net/jython/?rev=6503&view=rev
Author:   cgroves
Date:     2009-06-29 06:25:28 +0000 (Mon, 29 Jun 2009)

Log Message:
-----------
Explain how to set python.home for the context initializer

Modified Paths:
--------------
    trunk/jython/src/org/python/util/PyServlet.java
    trunk/jython/src/org/python/util/PyServletInitializer.java

Modified: trunk/jython/src/org/python/util/PyServlet.java
===================================================================
--- trunk/jython/src/org/python/util/PyServlet.java	2009-06-29 06:13:29 UTC (rev 6502)
+++ trunk/jython/src/org/python/util/PyServlet.java	2009-06-29 06:25:28 UTC (rev 6503)
@@ -50,7 +50,7 @@
  *         <servlet-class>org.python.util.PyServlet</servlet-class>
  *         <init-param>
  *             <param-name>python.home</param-name>
- *             <param-value>/usr/home/jython-2.1</param-value>
+ *             <param-value>/usr/home/jython-2.5</param-value>
  *         </init-param>
  *     </servlet>
  *     <servlet-mapping>

Modified: trunk/jython/src/org/python/util/PyServletInitializer.java
===================================================================
--- trunk/jython/src/org/python/util/PyServletInitializer.java	2009-06-29 06:13:29 UTC (rev 6502)
+++ trunk/jython/src/org/python/util/PyServletInitializer.java	2009-06-29 06:25:28 UTC (rev 6503)
@@ -16,6 +16,17 @@
  *       <load-on-startup>1</load-on-startup>
  *   </listener>
  *</pre>
+ *
+ * To use modules from Python's standard library in servlets and filters initialized by this
+ * listener, either add the standard library to the lib directory in WEB-INF, or add python.home as
+ * a context-param. The latter can be done by adding the following to web.xml:
+ *
+ * <pre>
+ * &lt;context-param&gt;
+ *     &lt;param-name&gt;python.home&lt;/param-name&gt;
+ *     &lt;param-value&gt;/usr/local/jython-2.5&lt;/param-value&gt;
+ * &lt;/context-param&gt;
+ * </pre>
  */
 public class PyServletInitializer implements ServletContextListener {
     public void contextInitialized(ServletContextEvent evt) {


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6502] trunk/jython/src/org/python/util

[<cg...@us...>]

Revision: 6502
          http://jython.svn.sourceforge.net/jython/?rev=6502&view=rev
Author:   cgroves
Date:     2009-06-29 06:13:29 +0000 (Mon, 29 Jun 2009)

Log Message:
-----------
Add the root dir and WEB-INF/jython to sys.path for PyFilters as well as PyServlets

Modified Paths:
--------------
    trunk/jython/src/org/python/util/PyFilter.java
    trunk/jython/src/org/python/util/PyServlet.java

Modified: trunk/jython/src/org/python/util/PyFilter.java
===================================================================
--- trunk/jython/src/org/python/util/PyFilter.java	2009-06-29 05:42:24 UTC (rev 6501)
+++ trunk/jython/src/org/python/util/PyFilter.java	2009-06-29 06:13:29 UTC (rev 6502)
@@ -11,7 +11,6 @@
 import javax.servlet.ServletResponse;
 
 import org.python.core.PyException;
-import org.python.core.PySystemState;
 import org.python.util.PythonInterpreter;
 
 /**
@@ -96,7 +95,7 @@
         if (!source.exists()) {
             throw new ServletException(source.getAbsolutePath() + " does not exist.");
         }
-        interp = new PythonInterpreter(null, new PySystemState());
+        interp = PyServlet.createInterpreter(config.getServletContext());
     }
 
     private String getRealPath(ServletContext context, String appPath) {

Modified: trunk/jython/src/org/python/util/PyServlet.java
===================================================================
--- trunk/jython/src/org/python/util/PyServlet.java	2009-06-29 05:42:24 UTC (rev 6501)
+++ trunk/jython/src/org/python/util/PyServlet.java	2009-06-29 06:13:29 UTC (rev 6502)
@@ -19,6 +19,7 @@
 import org.python.core.PyException;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PySystemState;
 
 /**
@@ -117,6 +118,17 @@
         PySystemState.add_extdir(rootPath + "WEB-INF" + File.separator + "lib", true);
     }
 
+    protected static PythonInterpreter createInterpreter(ServletContext servletContext) {
+        String rootPath = getRootPath(servletContext);
+        PySystemState sys = new PySystemState();
+        PythonInterpreter interp = new PythonInterpreter(new PyStringMap(), sys);
+        sys.path.append(new PyString(rootPath));
+
+        String modulesDir = rootPath + "WEB-INF" + File.separator + "jython";
+        sys.path.append(new PyString(modulesDir));
+        return interp;
+    }
+
     protected static String getRootPath(ServletContext context) {
         String rootPath = context.getRealPath("/");
         if (!rootPath.endsWith(File.separator)) {
@@ -154,15 +166,8 @@
      * requests.
      */
     public void reset() {
-        String rootPath = getRootPath(getServletContext());
         destroyCache();
-        interp = new PythonInterpreter(null, new PySystemState());
-
-        PySystemState sys = Py.getSystemState();
-        sys.path.append(new PyString(rootPath));
-
-        String modulesDir = rootPath + "WEB-INF" + File.separator + "jython";
-        sys.path.append(new PyString(modulesDir));
+        interp = createInterpreter(getServletContext());
     }
 
     private synchronized HttpServlet getServlet(String path)


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6501] trunk/jython/src/org/python/util/PyFilter.java

[<cg...@us...>]

Revision: 6501
          http://jython.svn.sourceforge.net/jython/?rev=6501&view=rev
Author:   cgroves
Date:     2009-06-29 05:42:24 +0000 (Mon, 29 Jun 2009)

Log Message:
-----------
Update error message for PyServlet not being a ServletContextListener

Modified Paths:
--------------
    trunk/jython/src/org/python/util/PyFilter.java

Modified: trunk/jython/src/org/python/util/PyFilter.java
===================================================================
--- trunk/jython/src/org/python/util/PyFilter.java	2009-06-29 02:23:57 UTC (rev 6500)
+++ trunk/jython/src/org/python/util/PyFilter.java	2009-06-29 05:42:24 UTC (rev 6501)
@@ -83,8 +83,9 @@
 
     public void init(FilterConfig config) throws ServletException {
         if (config.getServletContext().getAttribute(PyServlet.INIT_ATTR) == null) {
-            throw new ServletException("PyServlet has not been initialized, either as a servlet "
-                    + "or as context listener.  This must happen before PyFilter is initialized.");
+            throw new ServletException("Jython has not been initialized.  Add "
+                    + "org.python.util.PyServletInitializer as a listener to your "
+                    + "web.xml.");
         }
         this.config = config;
         String filterPath = config.getInitParameter(FILTER_PATH_PARAM);


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6500] trunk/jython

[<cg...@us...>]

Revision: 6500
          http://jython.svn.sourceforge.net/jython/?rev=6500&view=rev
Author:   cgroves
Date:     2009-06-29 02:23:57 +0000 (Mon, 29 Jun 2009)

Log Message:
-----------
Adding a test for bug #1381 that doesn't actually cause the problem described

Modified Paths:
--------------
    trunk/jython/Lib/test/test_java_visibility.py

Added Paths:
-----------
    trunk/jython/tests/java/org/python/tests/DisagreeingInterfaceOverrides.java

Modified: trunk/jython/Lib/test/test_java_visibility.py
===================================================================
--- trunk/jython/Lib/test/test_java_visibility.py	2009-06-23 07:50:56 UTC (rev 6499)
+++ trunk/jython/Lib/test/test_java_visibility.py	2009-06-29 02:23:57 UTC (rev 6500)
@@ -3,10 +3,11 @@
 import subprocess
 import sys
 from test import test_support
-from java.lang import Byte, Class
-from java.util import ArrayList, Collections, HashMap, Observable, Observer 
-from org.python.tests import (Coercions, HiddenSuper, InterfaceCombination, Invisible, Matryoshka,
-        OnlySubclassable, OtherSubVisible, SomePyMethods, SubVisible, Visible, VisibleOverride)
+from java.lang import Byte, Class, Integer
+from java.util import ArrayList, Collections, HashMap, LinkedList, Observable, Observer 
+from org.python.tests import (Coercions, DisagreeingInterfaceOverrides, HiddenSuper,
+        InterfaceCombination, Invisible, Matryoshka, OnlySubclassable, OtherSubVisible,
+        SomePyMethods, SubVisible, Visible, VisibleOverride)
 from org.python.tests import VisibilityResults as Results
 
 class VisibilityTest(unittest.TestCase):
@@ -155,6 +156,16 @@
         synchList.add("a string")
         self.assertEquals("a string", synchList.remove(0))
 
+    def test_interface_methods_merged(self):
+        '''Checks that implementing interfaces that use the same method name are all reachable.
+
+        Bug #1381'''
+        imp = DisagreeingInterfaceOverrides.Implementation()
+        self.assertEquals("String", imp.call("string argument"))
+        self.assertEquals("int", imp.call(Integer(7)))
+        self.assertEquals("List", imp.call(LinkedList()))
+        self.assertEquals("ArrayList", imp.call(ArrayList()))
+
 class JavaClassTest(unittest.TestCase):
     def test_class_methods_visible(self):
         self.assertFalse(HashMap.isInterface(),

Added: trunk/jython/tests/java/org/python/tests/DisagreeingInterfaceOverrides.java
===================================================================
--- trunk/jython/tests/java/org/python/tests/DisagreeingInterfaceOverrides.java	                        (rev 0)
+++ trunk/jython/tests/java/org/python/tests/DisagreeingInterfaceOverrides.java	2009-06-29 02:23:57 UTC (rev 6500)
@@ -0,0 +1,51 @@
+package org.python.tests;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Part of a test for issue #1381. It checks that Jython finds the proper overridden method when
+ * dealing with several interfaces. The test itself is in
+ * Lib/test_java_visibility.py#test_interface_methods_merged
+ */
+public class DisagreeingInterfaceOverrides {
+
+    public interface StringArg {
+
+        String call(String arg);
+    }
+
+    public interface IntArg {
+
+        String call(int arg);
+    }
+
+    public interface ListArg {
+
+        String call(List<Object> arg);
+    }
+
+    public interface ArrayListArg {
+
+        String call(List<Object> arg);
+    }
+
+    public static class Implementation implements StringArg, IntArg, ListArg, ArrayListArg {
+
+        public String call(String arg) {
+            return "String";
+        }
+
+        public String call(int arg) {
+            return "int";
+        }
+
+        public String call(List<Object> arg) {
+            return "List";
+        }
+
+        public String call(ArrayList<Object> arg) {
+            return "ArrayList";
+        }
+    }
+}


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6499] trunk/jython/NEWS

[<cg...@us...>]

Revision: 6499
          http://jython.svn.sourceforge.net/jython/?rev=6499&view=rev
Author:   cgroves
Date:     2009-06-23 07:50:56 +0000 (Tue, 23 Jun 2009)

Log Message:
-----------
NEWS updates

Modified Paths:
--------------
    trunk/jython/NEWS

Modified: trunk/jython/NEWS
===================================================================
--- trunk/jython/NEWS	2009-06-23 06:51:07 UTC (rev 6498)
+++ trunk/jython/NEWS	2009-06-23 07:50:56 UTC (rev 6499)
@@ -1,8 +1,11 @@
 Jython NEWS
 
 Jython 2.5.1
- New Features
+  New Features
     - Upgraded to ANTLR 3.1.3
+    - [ 1859477 ] Dynamically loaded ServletFilters like PyServlet
+  Bugs Fixed
+    - [ 1377 ] Event names shadowed by a field name on Java types leads to a NPE
 
 Jython 2.5.0
   The same as rc4.
@@ -10,7 +13,7 @@
 Jython 2.5.0 rc4
   Bugs fixed
     - [ 1354 ] core language failures in interactive interpreter
-    - [ 1358 ] Simple pogram fails to parse in Jython 2.5rc3, but parses OK with CPython
+    - [ 1358 ] Simple program fails to parse in Jython 2.5rc3, but parses OK with CPython
     - [ 1357 ] no sys.executable when script runner is a relative link
     - [ 1338 ] Comparing Java objects to each other fails when classes of values do not match
     - [ 1363 ] Deep inheritance from a Java class causes MRO problems


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6498] trunk/jython/src/org/python/core

[<cg...@us...>]

Revision: 6498
          http://jython.svn.sourceforge.net/jython/?rev=6498&view=rev
Author:   cgroves
Date:     2009-06-23 06:51:07 +0000 (Tue, 23 Jun 2009)

Log Message:
-----------
Don't extend PyReflectedField in PyBeanEventProperty.  Fixes Issue1377 by keeping PyJavaType from accessing field spuriously on PyBeanEventProperties.

Modified Paths:
--------------
    trunk/jython/src/org/python/core/PyBeanEventProperty.java
    trunk/jython/src/org/python/core/PyBeanProperty.java
    trunk/jython/src/org/python/core/PyJavaType.java
    trunk/jython/src/org/python/core/PyReflectedField.java

Modified: trunk/jython/src/org/python/core/PyBeanEventProperty.java
===================================================================
--- trunk/jython/src/org/python/core/PyBeanEventProperty.java	2009-06-23 06:21:59 UTC (rev 6497)
+++ trunk/jython/src/org/python/core/PyBeanEventProperty.java	2009-06-23 06:51:07 UTC (rev 6498)
@@ -9,7 +9,7 @@
 
 import org.python.util.Generic;
 
-public class PyBeanEventProperty extends PyReflectedField {
+public class PyBeanEventProperty extends PyObject {
 
     private static Map<String, Class<?>> adapterClasses = Generic.map();
 
@@ -38,6 +38,7 @@
         this.eventClass = eventClass;
     }
 
+    @Override
     public PyObject _doget(PyObject self) {
         if (self == null) {
             return this;
@@ -65,6 +66,7 @@
         return func;
     }
 
+    @Override
     public boolean _doset(PyObject self, PyObject value) {
         Object jself = Py.tojava(self, addMethod.getDeclaringClass());
         if (!(value instanceof PyCompoundCallable)) {
@@ -77,6 +79,7 @@
         return true;
     }
 
+    @Override
     public String toString() {
         return "<beanEventProperty " + __name__ + " for event " + eventClass.toString() + " "
                 + Py.idstr(this) + ">";

Modified: trunk/jython/src/org/python/core/PyBeanProperty.java
===================================================================
--- trunk/jython/src/org/python/core/PyBeanProperty.java	2009-06-23 06:21:59 UTC (rev 6497)
+++ trunk/jython/src/org/python/core/PyBeanProperty.java	2009-06-23 06:51:07 UTC (rev 6498)
@@ -14,6 +14,7 @@
         this.myType = myType;
     }
 
+    @Override
     public PyObject _doget(PyObject self) {
         if (self == null) {
             if (field != null) {
@@ -36,6 +37,7 @@
         }
     }
 
+    @Override
     public boolean _doset(PyObject self, PyObject value) {
         if (self == null) {
             if (field != null) {
@@ -72,6 +74,7 @@
         return new PyBeanProperty(__name__, myType, getMethod, setMethod);
     }
 
+    @Override
     public String toString() {
         String typeName = "unknown";
         if (myType != null) {

Modified: trunk/jython/src/org/python/core/PyJavaType.java
===================================================================
--- trunk/jython/src/org/python/core/PyJavaType.java	2009-06-23 06:21:59 UTC (rev 6497)
+++ trunk/jython/src/org/python/core/PyJavaType.java	2009-06-23 06:51:07 UTC (rev 6498)
@@ -56,8 +56,10 @@
         super(TYPE == null ? fromClass(PyType.class) : TYPE);
     }
 
+    @Override
     protected boolean useMetatypeFirst(PyObject attr) {
-        return !(attr instanceof PyReflectedField || attr instanceof PyReflectedFunction);
+        return !(attr instanceof PyReflectedField || attr instanceof PyReflectedFunction ||
+                attr instanceof PyBeanEventProperty);
     }
 
     // Java types are ok with things being added and removed from their dicts as long as there isn't

Modified: trunk/jython/src/org/python/core/PyReflectedField.java
===================================================================
--- trunk/jython/src/org/python/core/PyReflectedField.java	2009-06-23 06:21:59 UTC (rev 6497)
+++ trunk/jython/src/org/python/core/PyReflectedField.java	2009-06-23 06:51:07 UTC (rev 6498)
@@ -4,7 +4,6 @@
 import java.lang.reflect.Field;
 import java.lang.reflect.Modifier;
 
-
 public class PyReflectedField extends PyObject {
     public Field field;
 
@@ -14,6 +13,7 @@
         this.field = field;
     }
 
+    @Override
     public PyObject _doget(PyObject self) {
         Object iself = null;
         if (!Modifier.isStatic(field.getModifiers())) {
@@ -32,6 +32,7 @@
         return Py.java2py(value);
     }
 
+    @Override
     public boolean _doset(PyObject self, PyObject value) {
         Object iself = null;
         if (!Modifier.isStatic(field.getModifiers())) {
@@ -51,6 +52,7 @@
         return true;
     }
 
+    @Override
     public String toString() {
         return "<reflected field "+field.toString()+" "+Py.idstr(this)+">";
     }


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6497] trunk/jython/ACKNOWLEDGMENTS

[<cg...@us...>]

Revision: 6497
          http://jython.svn.sourceforge.net/jython/?rev=6497&view=rev
Author:   cgroves
Date:     2009-06-23 06:21:59 +0000 (Tue, 23 Jun 2009)

Log Message:
-----------
Adding jfeinberg for PyFilter

Modified Paths:
--------------
    trunk/jython/ACKNOWLEDGMENTS

Modified: trunk/jython/ACKNOWLEDGMENTS
===================================================================
--- trunk/jython/ACKNOWLEDGMENTS	2009-06-23 06:17:57 UTC (rev 6496)
+++ trunk/jython/ACKNOWLEDGMENTS	2009-06-23 06:21:59 UTC (rev 6497)
@@ -85,6 +85,7 @@
     Clark Updike
     Leonardo Soto
     James Robinson
+    Jonathan Feinberg
 
 Local Variables:
 mode: indented-text


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6496] trunk/jython/src/org/python/util

[<cg...@us...>]

Revision: 6496
          http://jython.svn.sourceforge.net/jython/?rev=6496&view=rev
Author:   cgroves
Date:     2009-06-23 06:17:57 +0000 (Tue, 23 Jun 2009)

Log Message:
-----------
Break an initializer clas out of PyServlet to keep PyServlet from being a ServletContextListener.

Modified Paths:
--------------
    trunk/jython/src/org/python/util/PyFilter.java
    trunk/jython/src/org/python/util/PyServlet.java

Added Paths:
-----------
    trunk/jython/src/org/python/util/PyServletInitializer.java

Modified: trunk/jython/src/org/python/util/PyFilter.java
===================================================================
--- trunk/jython/src/org/python/util/PyFilter.java	2009-06-23 02:20:07 UTC (rev 6495)
+++ trunk/jython/src/org/python/util/PyFilter.java	2009-06-23 06:17:57 UTC (rev 6496)
@@ -43,17 +43,16 @@
  * </p>
  *
  * <pre>
- * &lt;!-- PyFilter depends on PyServlet --&gt;
- * &lt;servlet&gt;
- *  &lt;servlet-name&gt;PyServlet&lt;/servlet-name&gt;
- *  &lt;servlet-class&gt;org.python.util.PyServlet&lt;/servlet-class&gt;
- *  &lt;load-on-startup&gt;1&lt;/load-on-startup&gt;
- * &lt;/servlet&gt;
+ * &lt;!-- Initialize the Jython runtime --&gt;
+ *   &lt;listener&gt;
+ *       &lt;listener-class&gt;org.python.util.PyServletInitializer&lt;/listener-class&gt;
+ *       &lt;load-on-startup&gt;1&lt;/load-on-startup&gt;
+ *   &lt;/listener&gt;
  *
  * &lt;!-- Declare a uniquely-named PyFilter --&gt;
  * &lt;filter&gt;
  *  &lt;filter-name&gt;HeaderFilter&lt;/filter-name&gt;
- *  &lt;filter-class&gt;org.jython.util.PyFilter&lt;/filter-class&gt;
+ *  &lt;filter-class&gt;org.python.util.PyFilter&lt;/filter-class&gt;
  *
  *  &lt;!-- The special param __filter__ gives the context-relative path to the Jython source file --&gt;
  *  &lt;init-param&gt;
@@ -72,20 +71,6 @@
  *  &lt;url-pattern&gt;/*&lt;/url-pattern&gt;
  * &lt;/filter-mapping&gt;
  * </pre>
- *
- * <p>
- * PyFilter depends on initialization code from PyServlet being run. If PyServlet is used to serve
- * pages, this code will be executed and PyFilter will work properly. However, if you aren't using
- * PyServlet, the initialization code can be invoked as a ServletContextListener instead of as an
- * HttpServlet. Use the following in web.xml instead of a servlet tag:
- *
- * <pre>
- *   &lt;listener&gt;
- *       &lt;listener-class&gt;org.python.util.PyServlet&lt;/listener-class&gt;
- *       &lt;load-on-startup&gt;1&lt;/load-on-startup&gt;
- *   &lt;/listener&gt;
- * </pre>
- *
  */
 public class PyFilter implements Filter {
     public static final String FILTER_PATH_PARAM = "__filter__";

Modified: trunk/jython/src/org/python/util/PyServlet.java
===================================================================
--- trunk/jython/src/org/python/util/PyServlet.java	2009-06-23 02:20:07 UTC (rev 6495)
+++ trunk/jython/src/org/python/util/PyServlet.java	2009-06-23 06:17:57 UTC (rev 6496)
@@ -9,8 +9,6 @@
 import java.util.regex.Pattern;
 
 import javax.servlet.ServletContext;
-import javax.servlet.ServletContextEvent;
-import javax.servlet.ServletContextListener;
 import javax.servlet.ServletException;
 import javax.servlet.ServletRequest;
 import javax.servlet.ServletResponse;
@@ -62,18 +60,12 @@
  *
  * </pre>
  */
-public class PyServlet extends HttpServlet implements ServletContextListener {
+public class PyServlet extends HttpServlet {
 
     public static final String SKIP_INIT_NAME = "skip_jython_initialization";
 
     protected static final String INIT_ATTR = "__jython_initialized__";
 
-    public void contextInitialized(ServletContextEvent event) {
-        init(new Properties(), event.getServletContext(), "a ServletContextListener", true);
-    }
-
-    public void contextDestroyed(ServletContextEvent event) {}
-
     @Override
     public void init() {
         Properties props = new Properties();
@@ -83,8 +75,18 @@
             String name = (String)e.nextElement();
             props.put(name, getInitParameter(name));
         }
-        init(props, getServletContext(), "the servlet " + getServletConfig().getServletName(),
-             getServletConfig().getInitParameter(SKIP_INIT_NAME) != null);
+        boolean initialize = getServletConfig().getInitParameter(SKIP_INIT_NAME) != null;
+        if (getServletContext().getAttribute(INIT_ATTR) != null) {
+            if (initialize) {
+                System.err.println("Jython has already been initialized in this context, not "
+                        + "initializing for " + getServletName() + ".  Add " + SKIP_INIT_NAME
+                        + " to as an init param to this servlet's configuration to indicate this "
+                        + "is expected.");
+            }
+        } else if (initialize) {
+            init(props, getServletContext());
+        }
+        reset();
     }
 
     /**
@@ -92,46 +94,37 @@
      * servlet, and this is the shared init code. If both initializations are used in a single
      * context, the system state initialization code only runs once.
      */
-    private void init(Properties props, ServletContext context, String initializerName,
-                      boolean initialize) {
-        rootPath = context.getRealPath("/");
+    protected static void init(Properties props, ServletContext context) {
+        String rootPath = getRootPath(context);
+        context.setAttribute(INIT_ATTR, true);
+        Properties baseProps = PySystemState.getBaseProperties();
+        // Context parameters
+        Enumeration<?> e = context.getInitParameterNames();
+        while (e.hasMoreElements()) {
+            String name = (String)e.nextElement();
+            props.put(name, context.getInitParameter(name));
+        }
+        if (props.getProperty("python.home") == null
+                && baseProps.getProperty("python.home") == null) {
+            props.put("python.home", rootPath + "WEB-INF" + File.separator + "lib");
+        }
+        PySystemState.initialize(baseProps, props, new String[0]);
+        PySystemState.add_package("javax.servlet");
+        PySystemState.add_package("javax.servlet.http");
+        PySystemState.add_package("javax.servlet.jsp");
+        PySystemState.add_package("javax.servlet.jsp.tagext");
+        PySystemState.add_classdir(rootPath + "WEB-INF" + File.separator + "classes");
+        PySystemState.add_extdir(rootPath + "WEB-INF" + File.separator + "lib", true);
+    }
+
+    protected static String getRootPath(ServletContext context) {
+        String rootPath = context.getRealPath("/");
         if (!rootPath.endsWith(File.separator)) {
             rootPath += File.separator;
         }
-        if (context.getAttribute(INIT_ATTR) != null) {
-            if (initialize) {
-                System.err.println("Jython has already been initialized by "
-                        + context.getAttribute(INIT_ATTR)
-                        + " in this context, not initializing for " + initializerName + ".  Add "
-                        + SKIP_INIT_NAME
-                        + " to as an init param to this servlet's configuration to indicate this "
-                        + "is expected.");
-            }
-        } else if (initialize) {
-            context.setAttribute(INIT_ATTR, initializerName);
-            Properties baseProps = PySystemState.getBaseProperties();
-            // Context parameters
-            Enumeration<?> e = context.getInitParameterNames();
-            while (e.hasMoreElements()) {
-                String name = (String)e.nextElement();
-                props.put(name, context.getInitParameter(name));
-            }
-            if (props.getProperty("python.home") == null
-                    && baseProps.getProperty("python.home") == null) {
-                props.put("python.home", rootPath + "WEB-INF" + File.separator + "lib");
-            }
-            PySystemState.initialize(baseProps, props, new String[0]);
-            PySystemState.add_package("javax.servlet");
-            PySystemState.add_package("javax.servlet.http");
-            PySystemState.add_package("javax.servlet.jsp");
-            PySystemState.add_package("javax.servlet.jsp.tagext");
-            PySystemState.add_classdir(rootPath + "WEB-INF" + File.separator + "classes");
-            PySystemState.add_extdir(rootPath + "WEB-INF" + File.separator + "lib", true);
-        }
-        reset();
+        return rootPath;
     }
 
-
     @Override
     public void service(ServletRequest req, ServletResponse res)
         throws ServletException, IOException
@@ -161,6 +154,7 @@
      * requests.
      */
     public void reset() {
+        String rootPath = getRootPath(getServletContext());
         destroyCache();
         interp = new PythonInterpreter(null, new PySystemState());
 
@@ -246,6 +240,5 @@
     private static final Pattern FIND_NAME = Pattern.compile("([^/]+)\\.py$");
 
     private PythonInterpreter interp;
-    private String rootPath;
     private Map<String, CacheEntry> cache = Generic.map();
 }
\ No newline at end of file

Added: trunk/jython/src/org/python/util/PyServletInitializer.java
===================================================================
--- trunk/jython/src/org/python/util/PyServletInitializer.java	                        (rev 0)
+++ trunk/jython/src/org/python/util/PyServletInitializer.java	2009-06-23 06:17:57 UTC (rev 6496)
@@ -0,0 +1,26 @@
+package org.python.util;
+
+import java.util.Properties;
+
+import javax.servlet.ServletContextEvent;
+import javax.servlet.ServletContextListener;
+
+/**
+ * Initializes the jython runtime inside a servlet engine. Should be used with {@link PyFilter} to
+ * initialize the system before the filter starts. Add the following to web.xml to run the
+ * initializer:
+ *
+ * <pre>
+ *   &lt;listener&gt;
+ *       &lt;listener-class&gt;org.python.util.PyServletInitializer&lt;/listener-class&gt;
+ *       &lt;load-on-startup&gt;1&lt;/load-on-startup&gt;
+ *   &lt;/listener&gt;
+ *</pre>
+ */
+public class PyServletInitializer implements ServletContextListener {
+    public void contextInitialized(ServletContextEvent evt) {
+        PyServlet.init(new Properties(), evt.getServletContext());
+    }
+
+    public void contextDestroyed(ServletContextEvent evt) {}
+}


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6495] trunk/jython/grammar/PythonPartial.g

[<fwi...@us...>]

Revision: 6495
          http://jython.svn.sourceforge.net/jython/?rev=6495&view=rev
Author:   fwierzbicki
Date:     2009-06-23 02:20:07 +0000 (Tue, 23 Jun 2009)

Log Message:
-----------
Part 1 of making PythonPartial.g diffable from Python.g.

Modified Paths:
--------------
    trunk/jython/grammar/PythonPartial.g

Modified: trunk/jython/grammar/PythonPartial.g
===================================================================
--- trunk/jython/grammar/PythonPartial.g	2009-06-22 16:38:12 UTC (rev 6494)
+++ trunk/jython/grammar/PythonPartial.g	2009-06-23 02:20:07 UTC (rev 6495)
@@ -52,14 +52,11 @@
  *
  *  I (Terence) tested this by running it on the jython-2.1/Lib
  *  directory of 40k lines of Python.
- *  
+ *
+ *  REQUIRES ANTLR v3
+ *
  *  Updated to Python 2.5 by Frank Wierzbicki.
  *
- *  This particular version has some changes to allow "partial" parsing
- *  So that an interactive session can tell if it should wait for more
- *  input or not.  For example, this grammar will allow a String that
- *  starts with """ but has no ending """ and will allow a Suite to have
- *  an indent but no dedent.
  */
 
 parser grammar PythonPartial;
@@ -93,58 +90,160 @@
 }
 }
 
-single_input : NEWLINE
-             | simple_stmt
-             | compound_stmt NEWLINE?
-             ;
+//single_input: NEWLINE | simple_stmt | compound_stmt NEWLINE
+single_input
 
+    : NEWLINE
+    | simple_stmt
+    | compound_stmt NEWLINE?
+    ;
+
 //eval_input: testlist NEWLINE* ENDMARKER
-eval_input : LEADING_WS? (NEWLINE)* testlist? (NEWLINE)* EOF
-           ;
+eval_input
 
-decorators: decorator+
-          ;
+    : LEADING_WS? (NEWLINE)* testlist? (NEWLINE)* EOF
+    ;
 
-decorator: AT dotted_attr (LPAREN arglist? RPAREN)? NEWLINE
-         ;
+//not in CPython's Grammar file
+dotted_attr 
+    : NAME
+      ( (DOT NAME)+ 
+      | 
+      )
+    ;
 
-dotted_attr
-    : NAME (DOT NAME)*
+//attr is here for Java  compatibility.  A Java foo.getIf() can be called from Jython as foo.if
+//     so we need to support any keyword as an attribute.
+
+attr
+    : NAME
+    | AND
+    | AS
+    | ASSERT
+    | BREAK
+    | CLASS
+    | CONTINUE
+    | DEF
+    | DELETE
+    | ELIF
+    | EXCEPT
+    | EXEC
+    | FINALLY
+    | FROM
+    | FOR
+    | GLOBAL
+    | IF
+    | IMPORT
+    | IN
+    | IS
+    | LAMBDA
+    | NOT
+    | OR
+    | ORELSE
+    | PASS
+    | PRINT
+    | RAISE
+    | RETURN
+    | TRY
+    | WHILE
+    | WITH
+    | YIELD
     ;
 
-funcdef : decorators? DEF NAME parameters COLON suite
-        ;
+//decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE
+decorator 
 
-parameters : LPAREN (varargslist)? RPAREN
-           ;
+    : AT dotted_attr 
+    ( LPAREN
+      ( arglist
+        
+      | 
+      )
+      RPAREN
+    | 
+    ) NEWLINE
+    ;
 
-varargslist : defparameter (options {greedy=true;}:COMMA defparameter)*
-              (COMMA
-                  ( STAR NAME (COMMA DOUBLESTAR NAME)?
-                  | DOUBLESTAR NAME
-                  )?
-              )?
-            | STAR NAME (COMMA DOUBLESTAR NAME)?
-            | DOUBLESTAR NAME
-            ;
+//decorators: decorator+
+decorators 
+    : decorator+
+      
+    ;
 
-defparameter : fpdef (ASSIGN test)?
-             ;
+//funcdef: [decorators] 'def' NAME parameters ':' suite
+funcdef
 
-fpdef : NAME
-      | LPAREN fplist RPAREN
-      ;
+    : decorators? DEF NAME parameters COLON suite
+    
+    ;
 
-fplist : fpdef (options {greedy=true;}:COMMA fpdef)* (COMMA)?
-       ;
+//parameters: '(' [varargslist] ')'
+parameters 
+    : LPAREN 
+      (varargslist 
+      | 
+      )
+      RPAREN
+    ;
 
-stmt : simple_stmt
-     | compound_stmt
-     ;
+//not in CPython's Grammar file
+defparameter 
 
-simple_stmt : small_stmt (options {greedy=true;}:SEMI small_stmt)* (SEMI)? (NEWLINE|EOF)
-            ;
+    : fpdef (ASSIGN test)?
+      
+    ;
 
+//varargslist: ((fpdef ['=' test] ',')*
+//              ('*' NAME [',' '**' NAME] | '**' NAME) |
+//              fpdef ['=' test] (',' fpdef ['=' test])* [','])
+varargslist 
+
+    : defparameter (options {greedy=true;}:COMMA defparameter)*
+      (COMMA
+          (STAR NAME (COMMA DOUBLESTAR NAME)?
+          | DOUBLESTAR NAME
+          )?
+      )?
+      
+    | STAR NAME (COMMA DOUBLESTAR NAME)?
+      
+    | DOUBLESTAR NAME
+      
+    ;
+
+//fpdef: NAME | '(' fplist ')'
+fpdef
+    : NAME 
+   
+    | (LPAREN fpdef COMMA) => LPAREN fplist RPAREN
+   
+    | LPAREN fplist RPAREN
+   
+    ;
+
+//fplist: fpdef (',' fpdef)* [',']
+fplist 
+    : fpdef
+      (options {greedy=true;}:COMMA fpdef)* (COMMA)?
+      
+    ;
+
+//stmt: simple_stmt | compound_stmt
+stmt 
+    : simple_stmt
+      
+    | compound_stmt
+      
+    ;
+
+//simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE
+simple_stmt 
+    : small_stmt (options {greedy=true;}:SEMI small_stmt)* (SEMI)? (NEWLINE|EOF)
+      
+    ;
+
+//small_stmt: (expr_stmt | print_stmt  | del_stmt | pass_stmt | flow_stmt |
+//             import_stmt | global_stmt | exec_stmt | assert_stmt)
 small_stmt : expr_stmt
            | print_stmt
            | del_stmt


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Jython-checkins] SF.net SVN: jython:[6494] trunk/jython

[<fwi...@us...>]

Revision: 6494
          http://jython.svn.sourceforge.net/jython/?rev=6494&view=rev
Author:   fwierzbicki
Date:     2009-06-22 16:38:12 +0000 (Mon, 22 Jun 2009)

Log Message:
-----------
Upgraded to ANTLR 3.1.3

Modified Paths:
--------------
    trunk/jython/.classpath
    trunk/jython/NEWS
    trunk/jython/build.xml

Added Paths:
-----------
    trunk/jython/extlibs/antlr-3.1.3.jar
    trunk/jython/extlibs/antlr-runtime-3.1.3.jar

Removed Paths:
-------------
    trunk/jython/extlibs/antlr-3.1.2.jar
    trunk/jython/extlibs/antlr-runtime-3.1.2.jar

Modified: trunk/jython/.classpath
===================================================================
--- trunk/jython/.classpath	2009-06-22 08:43:07 UTC (rev 6493)
+++ trunk/jython/.classpath	2009-06-22 16:38:12 UTC (rev 6494)
@@ -14,7 +14,7 @@
 	<classpathentry kind="lib" path="extlibs/postgresql-8.3-603.jdbc4.jar"/>
 	<classpathentry kind="lib" path="extlibs/servlet-api-2.5.jar"/>
 	<classpathentry kind="var" path="ANT_HOME/lib/ant.jar"/>
-	<classpathentry kind="lib" path="extlibs/antlr-runtime-3.1.2.jar"/>
+	<classpathentry kind="lib" path="extlibs/antlr-runtime-3.1.3.jar"/>
 	<classpathentry kind="lib" path="extlibs/asm-3.1.jar"/>
 	<classpathentry kind="lib" path="extlibs/asm-commons-3.1.jar"/>
 	<classpathentry kind="lib" path="extlibs/constantine-0.4.jar"/>

Modified: trunk/jython/NEWS
===================================================================
--- trunk/jython/NEWS	2009-06-22 08:43:07 UTC (rev 6493)
+++ trunk/jython/NEWS	2009-06-22 16:38:12 UTC (rev 6494)
@@ -1,5 +1,9 @@
 Jython NEWS
 
+Jython 2.5.1
+ New Features
+    - Upgraded to ANTLR 3.1.3
+
 Jython 2.5.0
   The same as rc4.
 

Modified: trunk/jython/build.xml
===================================================================
--- trunk/jython/build.xml	2009-06-22 08:43:07 UTC (rev 6493)
+++ trunk/jython/build.xml	2009-06-22 16:38:12 UTC (rev 6494)
@@ -152,7 +152,7 @@
             <pathelement path="${extlibs.dir}/mysql-connector-java-5.1.6.jar" />
             <pathelement path="${extlibs.dir}/postgresql-8.3-603.jdbc4.jar" />
             <pathelement path="${extlibs.dir}/antlr-2.7.7.jar" />
-            <pathelement path="${extlibs.dir}/antlr-3.1.2.jar" />
+            <pathelement path="${extlibs.dir}/antlr-3.1.3.jar" />
             <pathelement path="${extlibs.dir}/stringtemplate-3.2.jar" />
 
             <pathelement path="${extlibs.dir}/asm-3.1.jar" />
@@ -546,7 +546,7 @@
         <taskdef name="jarjar" classname="com.tonicsystems.jarjar.JarJarTask" classpath="extlibs/jarjar-0.7.jar"/>
         <jarjar destfile="${dist.dir}/${jython.deploy.jar}">
             <zipfileset src="${dist.dir}/${jython.dev.jar}"/>
-            <zipfileset src="extlibs/antlr-runtime-3.1.2.jar"/>
+            <zipfileset src="extlibs/antlr-runtime-3.1.3.jar"/>
             <rule pattern="org.antlr.runtime.**" result="org.python.antlr.runtime.@1"/>
             <zipfileset src="extlibs/asm-3.1.jar"/>
             <zipfileset src="extlibs/asm-commons-3.1.jar"/>

Deleted: trunk/jython/extlibs/antlr-3.1.2.jar
===================================================================
(Binary files differ)

Added: trunk/jython/extlibs/antlr-3.1.3.jar
===================================================================
(Binary files differ)


Property changes on: trunk/jython/extlibs/antlr-3.1.3.jar
___________________________________________________________________
Added: svn:mime-type
   + application/octet-stream

Deleted: trunk/jython/extlibs/antlr-runtime-3.1.2.jar
===================================================================
(Binary files differ)

Added: trunk/jython/extlibs/antlr-runtime-3.1.3.jar
===================================================================
(Binary files differ)


Property changes on: trunk/jython/extlibs/antlr-runtime-3.1.3.jar
___________________________________________________________________
Added: svn:mime-type
   + application/octet-stream


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.