Menu
▾
▴
[Jython-checkins] SF.net SVN: jython:[6207] branches/newlist
|
From: <fwi...@us...> - 2009-04-10 13:46:00
|
Revision: 6207
http://jython.svn.sourceforge.net/jython/?rev=6207&view=rev
Author: fwierzbicki
Date: 2009-04-10 13:45:27 +0000 (Fri, 10 Apr 2009)
Log Message:
-----------
Merged revisions 6161-6167,6169-6175,6177-6185,6187-6189,6193-6194,6198,6203-6205 via svnmerge from
https://jython.svn.sourceforge.net/svnroot/jython/trunk/jython
........
r6161 | pjenvey | 2009-04-04 21:53:25 -0400 (Sat, 04 Apr 2009) | 2 lines
convert PyClass to exposed annotations
........
r6162 | pjenvey | 2009-04-04 22:32:50 -0400 (Sat, 04 Apr 2009) | 2 lines
remove the _new.classobj hack now that PyClass __new__ does the right thing
........
r6163 | pjenvey | 2009-04-04 23:18:39 -0400 (Sat, 04 Apr 2009) | 2 lines
remove stop_at_java cruft -- remnants of old Java integration
........
r6164 | thobes | 2009-04-05 13:22:40 -0400 (Sun, 05 Apr 2009) | 2 lines
Fix for http://bugs.jython.org/issue1110
........
r6165 | fwierzbicki | 2009-04-05 14:29:58 -0400 (Sun, 05 Apr 2009) | 9 lines
Fix for http://bugs.jython.org/issue1264 'is not' test exhibits incorrect
behaviour when wrapping Java objects
Included is a reshuffling of the patch test so that a regression test can pick
it up. It isn't perfect since I had to use raw "assert" statements in the
python, but it is run by the Java unit test IdentityTest.
Thanks to Geoffrey French for the patch.
........
r6166 | pjenvey | 2009-04-05 14:42:28 -0400 (Sun, 05 Apr 2009) | 2 lines
fix ant javatest regrtest (the buildbot) unnecessarily compiling twice
........
r6167 | fwierzbicki | 2009-04-05 15:15:19 -0400 (Sun, 05 Apr 2009) | 2 lines
remove sys.path appends that where not needed.
........
r6169 | pjenvey | 2009-04-05 18:13:33 -0400 (Sun, 05 Apr 2009) | 2 lines
fix dictproxy equality checks
........
r6170 | pjenvey | 2009-04-05 19:21:56 -0400 (Sun, 05 Apr 2009) | 2 lines
type shouldn't actually have a __str__
........
r6171 | pjenvey | 2009-04-05 21:03:47 -0400 (Sun, 05 Apr 2009) | 1 line
make exposed methods final
........
r6172 | pjenvey | 2009-04-05 21:38:32 -0400 (Sun, 05 Apr 2009) | 2 lines
convert PyInstance to exposed annotations. fixes its binop rule problems (#1197)
........
r6173 | pjenvey | 2009-04-05 22:35:31 -0400 (Sun, 05 Apr 2009) | 3 lines
allow comparison of builtin methods, fixes test_descr.methodwrapper minus its
implementation detail
........
r6174 | pjenvey | 2009-04-05 23:12:07 -0400 (Sun, 05 Apr 2009) | 3 lines
avoid __tojava__ conversion to int in ldexp so it'll acutally raise
OverflowErors. helps out SymPy
........
r6175 | pjenvey | 2009-04-05 23:19:30 -0400 (Sun, 05 Apr 2009) | 1 line
fix complex.__nonzero__
........
r6177 | fwierzbicki | 2009-04-06 19:52:09 -0400 (Mon, 06 Apr 2009) | 2 lines
Added some missed contributors to ACKNOWLEDGMENTS file.
........
r6178 | fwierzbicki | 2009-04-06 20:33:46 -0400 (Mon, 06 Apr 2009) | 2 lines
Added missing committers to Acknowledgments file.
........
r6179 | pjenvey | 2009-04-06 20:37:45 -0400 (Mon, 06 Apr 2009) | 2 lines
these now match CPythonLib's
........
r6180 | pjenvey | 2009-04-06 21:49:15 -0400 (Mon, 06 Apr 2009) | 1 line
shadow sys.platform
........
r6181 | pjenvey | 2009-04-06 21:52:46 -0400 (Mon, 06 Apr 2009) | 2 lines
str/unicode don't need __unicode__, and having it actually confuses some code
........
r6182 | zyasoft | 2009-04-06 22:23:13 -0400 (Mon, 06 Apr 2009) | 2 lines
Changed my name to "Jim Baker"
........
r6183 | fwierzbicki | 2009-04-06 22:32:17 -0400 (Mon, 06 Apr 2009) | 3 lines
By Jim Baker's suggestion (I agree of course!) specifically acknowledging Alan's
contribution of the long standing external project Modjy.
........
r6184 | fwierzbicki | 2009-04-06 22:44:38 -0400 (Mon, 06 Apr 2009) | 2 lines
Adding the committer Leonardo Soto.
........
r6185 | pjenvey | 2009-04-07 01:48:59 -0400 (Tue, 07 Apr 2009) | 2 lines
parser string input shouldn't go through universal newlines mode
........
r6187 | pjenvey | 2009-04-07 21:14:13 -0400 (Tue, 07 Apr 2009) | 4 lines
from:
http://svn.python.org/projects/python/branches/release25-maint/Lib/compiler/pycodegen.py@53576
http://svn.python.org/projects/python/branches/release25-maint/Lib/compiler/transformer.py@53576
........
r6188 | pjenvey | 2009-04-07 21:23:13 -0400 (Tue, 07 Apr 2009) | 3 lines
o reapply our compiler workarounds from r4114
o support compiler.compile via builtin compile
........
r6189 | pjenvey | 2009-04-07 21:27:07 -0400 (Tue, 07 Apr 2009) | 3 lines
relax the Windows platform matching, don't bother special casing ce
fixes #1162
........
r6193 | pjenvey | 2009-04-08 21:26:22 -0400 (Wed, 08 Apr 2009) | 1 line
rearrange per coding standards
........
r6194 | pjenvey | 2009-04-09 21:09:14 -0400 (Thu, 09 Apr 2009) | 2 lines
remove 2.3 workarounds
........
r6198 | pjenvey | 2009-04-09 23:32:30 -0400 (Thu, 09 Apr 2009) | 5 lines
o cleanup/refactor PyType.newType and add __weakref__ support
fixes #1200
o actually encode unicode slot names
o ensure __doc__ on all types
........
r6203 | pjenvey | 2009-04-10 00:27:38 -0400 (Fri, 10 Apr 2009) | 1 line
test_pyexpat is fail
........
r6204 | pjenvey | 2009-04-10 01:01:22 -0400 (Fri, 10 Apr 2009) | 3 lines
fix a new GlobalRef being tracked (reaped) for every
getweakrefcount/getweakrefs query
........
r6205 | pjenvey | 2009-04-10 02:33:04 -0400 (Fri, 10 Apr 2009) | 2 lines
use the types module instead of the pending deprecation new, whitespace
........
Modified Paths:
--------------
branches/newlist/ACKNOWLEDGMENTS
branches/newlist/CoreExposed.includes
branches/newlist/Lib/compiler/pycodegen.py
branches/newlist/Lib/compiler/transformer.py
branches/newlist/Lib/new.py
branches/newlist/Lib/os.py
branches/newlist/Lib/subprocess.py
branches/newlist/Lib/test/regrtest.py
branches/newlist/Lib/test/test_builtin_jy.py
branches/newlist/Lib/test/test_class_jy.py
branches/newlist/Lib/test/test_codeop_jy.py
branches/newlist/Lib/test/test_complex_jy.py
branches/newlist/Lib/test/test_descr.py
branches/newlist/Lib/test/test_descr_jy.py
branches/newlist/Lib/test/test_dictproxy_jy.py
branches/newlist/Lib/test/test_slots_jy.py
branches/newlist/Misc/make_binops.py
branches/newlist/NEWS
branches/newlist/build.xml
branches/newlist/src/org/python/core/ParserFacade.java
branches/newlist/src/org/python/core/Py.java
branches/newlist/src/org/python/core/PyBuiltinMethod.java
branches/newlist/src/org/python/core/PyClass.java
branches/newlist/src/org/python/core/PyComplex.java
branches/newlist/src/org/python/core/PyDictProxy.java
branches/newlist/src/org/python/core/PyInstance.java
branches/newlist/src/org/python/core/PyObject.java
branches/newlist/src/org/python/core/PyString.java
branches/newlist/src/org/python/core/PySystemState.java
branches/newlist/src/org/python/core/PyType.java
branches/newlist/src/org/python/core/PyUnicode.java
branches/newlist/src/org/python/modules/Setup.java
branches/newlist/src/org/python/modules/_weakref/GlobalRef.java
branches/newlist/src/org/python/modules/_weakref/ReferenceType.java
branches/newlist/src/org/python/modules/_weakref/WeakrefModule.java
branches/newlist/src/org/python/modules/math.java
Added Paths:
-----------
branches/newlist/tests/java/org/python/tests/identity/
branches/newlist/tests/java/org/python/tests/identity/IdentityObject.java
branches/newlist/tests/java/org/python/tests/identity/IdentityTest.java
branches/newlist/tests/python/
branches/newlist/tests/python/identity_test.py
Removed Paths:
-------------
branches/newlist/Lib/test/cfgparser.1
branches/newlist/Lib/test/test_doctest.py
branches/newlist/src/org/python/modules/_newmodule.java
branches/newlist/tests/java/org/python/tests/identity/IdentityObject.java
branches/newlist/tests/java/org/python/tests/identity/IdentityTest.java
branches/newlist/tests/python/identity_test.py
Property Changed:
----------------
branches/newlist/
Property changes on: branches/newlist
___________________________________________________________________
Modified: svnmerge-integrated
- /branches/modjy:1-6074 /branches/pbcvm:1-6045 /trunk/jython:1-6159
+ /branches/modjy:1-6074 /branches/pbcvm:1-6045 /trunk/jython:1-6206
Modified: branches/newlist/ACKNOWLEDGMENTS
===================================================================
--- branches/newlist/ACKNOWLEDGMENTS 2009-04-10 12:51:05 UTC (rev 6206)
+++ branches/newlist/ACKNOWLEDGMENTS 2009-04-10 13:45:27 UTC (rev 6207)
@@ -31,6 +31,8 @@
Cyrille Morvan has written the code for the Jythonc ant task.
+ Alan Kennedy contributed modjy, which bridges WSGI to the Servlet API
+
A huge thanks goes to all the members of the jpython/jython
mailing lists. Other folks who have contributed to JPython and
Jython in ways large and small, in no particular order:
@@ -66,6 +68,20 @@
Nathan Franzen
Aleks Totic
Randolph Brown
+ Geoffrey French
+ Tobias Ivarsson
+ Lino Mastrodomenico
+ S\xE9bastien Boisg\xE9rault
+ Jim Baker
+ Charlie Groves
+ Otmar Humbel
+ Philip Jenvey
+ Nicholas Riley
+ Frank Wierzbicki
+ Khalid Zuberi
+ Sean McGrath
+ Clark Updike
+ Leonardo Soto
Local Variables:
mode: indented-text
Modified: branches/newlist/CoreExposed.includes
===================================================================
--- branches/newlist/CoreExposed.includes 2009-04-10 12:51:05 UTC (rev 6206)
+++ branches/newlist/CoreExposed.includes 2009-04-10 13:45:27 UTC (rev 6207)
@@ -6,6 +6,7 @@
org/python/core/PyBoolean.class
org/python/core/PyBuiltinCallable.class
org/python/core/PyCell.class
+org/python/core/PyClass.class
org/python/core/PyClassMethod.class
org/python/core/PyClassMethodDescr.class
org/python/core/PyComplex.class
@@ -19,6 +20,7 @@
org/python/core/PyFrozenSet.class
org/python/core/PyFunction.class
org/python/core/PyGenerator.class
+org/python/core/PyInstance.class
org/python/core/PyInteger.class
org/python/core/PyList.class
org/python/core/PyLong.class
Modified: branches/newlist/Lib/compiler/pycodegen.py
===================================================================
--- branches/newlist/Lib/compiler/pycodegen.py 2009-04-10 12:51:05 UTC (rev 6206)
+++ branches/newlist/Lib/compiler/pycodegen.py 2009-04-10 13:45:27 UTC (rev 6207)
@@ -3,17 +3,19 @@
import marshal
import struct
import sys
-import types
from cStringIO import StringIO
+is_jython = sys.platform.startswith('java')
from compiler import ast, parse, walk, syntax
-#from compiler import pyassem, misc, future, symbols
from compiler import misc, future, symbols
from compiler.consts import SC_LOCAL, SC_GLOBAL, SC_FREE, SC_CELL
-from compiler.consts import CO_VARARGS, CO_VARKEYWORDS, CO_NEWLOCALS,\
- CO_NESTED, CO_GENERATOR, CO_GENERATOR_ALLOWED, CO_FUTURE_DIVISION
-#from compiler.pyassem import TupleArg
-TupleArg = None
+from compiler.consts import (CO_VARARGS, CO_VARKEYWORDS, CO_NEWLOCALS,
+ CO_NESTED, CO_GENERATOR, CO_FUTURE_DIVISION,
+ CO_FUTURE_ABSIMPORT, CO_FUTURE_WITH_STATEMENT)
+if not is_jython:
+ from compiler.pyassem import TupleArg
+else:
+ TupleArg = None
# XXX The version-specific code can go, since this code only works with 2.x.
# Do we have Python 1.x or Python 2.x?
@@ -49,22 +51,26 @@
mod.dump(f)
f.close()
-def compile(source, filename, mode, flags=None, dont_inherit=None):
- """Replacement for builtin compile() function"""
- if flags is not None or dont_inherit is not None:
- raise RuntimeError, "not implemented yet"
+if is_jython:
+ # use __builtin__ compile
+ compile = compile
+else:
+ def compile(source, filename, mode, flags=None, dont_inherit=None):
+ """Replacement for builtin compile() function"""
+ if flags is not None or dont_inherit is not None:
+ raise RuntimeError, "not implemented yet"
- if mode == "single":
- gen = Interactive(source, filename)
- elif mode == "exec":
- gen = Module(source, filename)
- elif mode == "eval":
- gen = Expression(source, filename)
- else:
- raise ValueError("compile() 3rd arg must be 'exec' or "
- "'eval' or 'single'")
- gen.compile()
- return gen.code
+ if mode == "single":
+ gen = Interactive(source, filename)
+ elif mode == "exec":
+ gen = Module(source, filename)
+ elif mode == "eval":
+ gen = Expression(source, filename)
+ else:
+ raise ValueError("compile() 3rd arg must be 'exec' or "
+ "'eval' or 'single'")
+ gen.compile()
+ return gen.code
class AbstractCompileMode:
@@ -121,8 +127,7 @@
f.write(self.getPycHeader())
marshal.dump(self.code, f)
- #MAGIC = imp.get_magic()
- MAGIC = None
+ MAGIC = None if is_jython else imp.get_magic()
def getPycHeader(self):
# compile.c uses marshal to write a long directly, with
@@ -207,8 +212,6 @@
self.checkClass()
self.locals = misc.Stack()
self.setups = misc.Stack()
- self.curStack = 0
- self.maxStack = 0
self.last_lineno = None
self._setupGraphDelegation()
self._div_op = "BINARY_DIVIDE"
@@ -219,8 +222,10 @@
if feature == "division":
self.graph.setFlag(CO_FUTURE_DIVISION)
self._div_op = "BINARY_TRUE_DIVIDE"
- elif feature == "generators":
- self.graph.setFlag(CO_GENERATOR_ALLOWED)
+ elif feature == "absolute_import":
+ self.graph.setFlag(CO_FUTURE_ABSIMPORT)
+ elif feature == "with_statement":
+ self.graph.setFlag(CO_FUTURE_WITH_STATEMENT)
def initClass(self):
"""This method is called once for each class"""
@@ -371,6 +376,13 @@
self._visitFuncOrLambda(node, isLambda=1)
def _visitFuncOrLambda(self, node, isLambda=0):
+ if not isLambda and node.decorators:
+ for decorator in node.decorators.nodes:
+ self.visit(decorator)
+ ndecorators = len(node.decorators.nodes)
+ else:
+ ndecorators = 0
+
gen = self.FunctionGen(node, self.scopes, isLambda,
self.class_name, self.get_module())
walk(node.code, gen)
@@ -378,15 +390,9 @@
self.set_lineno(node)
for default in node.defaults:
self.visit(default)
- frees = gen.scope.get_free_vars()
- if frees:
- for name in frees:
- self.emit('LOAD_CLOSURE', name)
- self.emit('LOAD_CONST', gen)
- self.emit('MAKE_CLOSURE', len(node.defaults))
- else:
- self.emit('LOAD_CONST', gen)
- self.emit('MAKE_FUNCTION', len(node.defaults))
+ self._makeClosure(gen, len(node.defaults))
+ for i in range(ndecorators):
+ self.emit('CALL_FUNCTION', 1)
def visitClass(self, node):
gen = self.ClassGen(node, self.scopes,
@@ -398,14 +404,7 @@
for base in node.bases:
self.visit(base)
self.emit('BUILD_TUPLE', len(node.bases))
- frees = gen.scope.get_free_vars()
- for name in frees:
- self.emit('LOAD_CLOSURE', name)
- self.emit('LOAD_CONST', gen)
- if frees:
- self.emit('MAKE_CLOSURE', 0)
- else:
- self.emit('MAKE_FUNCTION', 0)
+ self._makeClosure(gen, 0)
self.emit('CALL_FUNCTION', 0)
self.emit('BUILD_CLASS')
self.storeName(node.name)
@@ -539,6 +538,19 @@
def visitOr(self, node):
self.visitTest(node, 'JUMP_IF_TRUE')
+ def visitIfExp(self, node):
+ endblock = self.newBlock()
+ elseblock = self.newBlock()
+ self.visit(node.test)
+ self.emit('JUMP_IF_FALSE', elseblock)
+ self.emit('POP_TOP')
+ self.visit(node.then)
+ self.emit('JUMP_FORWARD', endblock)
+ self.nextBlock(elseblock)
+ self.emit('POP_TOP')
+ self.visit(node.else_)
+ self.nextBlock(endblock)
+
def visitCompare(self, node):
self.visit(node.expr)
cleanup = self.newBlock()
@@ -624,33 +636,116 @@
self.newBlock()
self.emit('POP_TOP')
- # exception related
+ def _makeClosure(self, gen, args):
+ frees = gen.scope.get_free_vars()
+ if frees:
+ for name in frees:
+ self.emit('LOAD_CLOSURE', name)
+ self.emit('BUILD_TUPLE', len(frees))
+ self.emit('LOAD_CONST', gen)
+ self.emit('MAKE_CLOSURE', args)
+ else:
+ self.emit('LOAD_CONST', gen)
+ self.emit('MAKE_FUNCTION', args)
- def visitAssert(self, node):
- # XXX would be interesting to implement this via a
- # transformation of the AST before this stage
+ def visitGenExpr(self, node):
+ gen = GenExprCodeGenerator(node, self.scopes, self.class_name,
+ self.get_module())
+ walk(node.code, gen)
+ gen.finish()
+ self.set_lineno(node)
+ self._makeClosure(gen, 0)
+ # precomputation of outmost iterable
+ self.visit(node.code.quals[0].iter)
+ self.emit('GET_ITER')
+ self.emit('CALL_FUNCTION', 1)
+
+ def visitGenExprInner(self, node):
+ self.set_lineno(node)
+ # setup list
+
+ stack = []
+ for i, for_ in zip(range(len(node.quals)), node.quals):
+ start, anchor, end = self.visit(for_)
+ cont = None
+ for if_ in for_.ifs:
+ if cont is None:
+ cont = self.newBlock()
+ self.visit(if_, cont)
+ stack.insert(0, (start, cont, anchor, end))
+
+ self.visit(node.expr)
+ self.emit('YIELD_VALUE')
+ self.emit('POP_TOP')
+
+ for start, cont, anchor, end in stack:
+ if cont:
+ skip_one = self.newBlock()
+ self.emit('JUMP_FORWARD', skip_one)
+ self.startBlock(cont)
+ self.emit('POP_TOP')
+ self.nextBlock(skip_one)
+ self.emit('JUMP_ABSOLUTE', start)
+ self.startBlock(anchor)
+ self.emit('POP_BLOCK')
+ self.setups.pop()
+ self.startBlock(end)
+
+ self.emit('LOAD_CONST', None)
+
+ def visitGenExprFor(self, node):
+ start = self.newBlock()
+ anchor = self.newBlock()
end = self.newBlock()
- self.set_lineno(node)
- # XXX __debug__ and AssertionError appear to be special cases
- # -- they are always loaded as globals even if there are local
- # names. I guess this is a sort of renaming op.
- self.emit('LOAD_GLOBAL', '__debug__')
- self.emit('JUMP_IF_FALSE', end)
+
+ self.setups.push((LOOP, start))
+ self.emit('SETUP_LOOP', end)
+
+ if node.is_outmost:
+ self.loadName('.0')
+ else:
+ self.visit(node.iter)
+ self.emit('GET_ITER')
+
+ self.nextBlock(start)
+ self.set_lineno(node, force=True)
+ self.emit('FOR_ITER', anchor)
self.nextBlock()
- self.emit('POP_TOP')
+ self.visit(node.assign)
+ return start, anchor, end
+
+ def visitGenExprIf(self, node, branch):
+ self.set_lineno(node, force=True)
self.visit(node.test)
- self.emit('JUMP_IF_TRUE', end)
- self.nextBlock()
+ self.emit('JUMP_IF_FALSE', branch)
+ self.newBlock()
self.emit('POP_TOP')
- self.emit('LOAD_GLOBAL', 'AssertionError')
- if node.fail:
- self.visit(node.fail)
- self.emit('RAISE_VARARGS', 2)
- else:
- self.emit('RAISE_VARARGS', 1)
- self.nextBlock(end)
- self.emit('POP_TOP')
+ # exception related
+
+ def visitAssert(self, node):
+ # XXX would be interesting to implement this via a
+ # transformation of the AST before this stage
+ if __debug__:
+ end = self.newBlock()
+ self.set_lineno(node)
+ # XXX AssertionError appears to be special case -- it is always
+ # loaded as a global even if there is a local name. I guess this
+ # is a sort of renaming op.
+ self.nextBlock()
+ self.visit(node.test)
+ self.emit('JUMP_IF_TRUE', end)
+ self.nextBlock()
+ self.emit('POP_TOP')
+ self.emit('LOAD_GLOBAL', 'AssertionError')
+ if node.fail:
+ self.visit(node.fail)
+ self.emit('RAISE_VARARGS', 2)
+ else:
+ self.emit('RAISE_VARARGS', 1)
+ self.nextBlock(end)
+ self.emit('POP_TOP')
+
def visitRaise(self, node):
self.set_lineno(node)
n = 0
@@ -732,6 +827,45 @@
self.emit('END_FINALLY')
self.setups.pop()
+ __with_count = 0
+
+ def visitWith(self, node):
+ body = self.newBlock()
+ final = self.newBlock()
+ exitvar = "$exit%d" % self.__with_count
+ valuevar = "$value%d" % self.__with_count
+ self.__with_count += 1
+ self.set_lineno(node)
+ self.visit(node.expr)
+ self.emit('DUP_TOP')
+ self.emit('LOAD_ATTR', '__exit__')
+ self._implicitNameOp('STORE', exitvar)
+ self.emit('LOAD_ATTR', '__enter__')
+ self.emit('CALL_FUNCTION', 0)
+ if node.vars is None:
+ self.emit('POP_TOP')
+ else:
+ self._implicitNameOp('STORE', valuevar)
+ self.emit('SETUP_FINALLY', final)
+ self.nextBlock(body)
+ self.setups.push((TRY_FINALLY, body))
+ if node.vars is not None:
+ self._implicitNameOp('LOAD', valuevar)
+ self._implicitNameOp('DELETE', valuevar)
+ self.visit(node.vars)
+ self.visit(node.body)
+ self.emit('POP_BLOCK')
+ self.setups.pop()
+ self.emit('LOAD_CONST', None)
+ self.nextBlock(final)
+ self.setups.push((END_FINALLY, final))
+ self._implicitNameOp('LOAD', exitvar)
+ self._implicitNameOp('DELETE', exitvar)
+ self.emit('WITH_CLEANUP')
+ self.emit('END_FINALLY')
+ self.setups.pop()
+ self.__with_count -= 1
+
# misc
def visitDiscard(self, node):
@@ -759,8 +893,10 @@
def visitImport(self, node):
self.set_lineno(node)
+ level = 0 if self.graph.checkFlag(CO_FUTURE_ABSIMPORT) else -1
for name, alias in node.names:
if VERSION > 1:
+ self.emit('LOAD_CONST', level)
self.emit('LOAD_CONST', None)
self.emit('IMPORT_NAME', name)
mod = name.split(".")[0]
@@ -772,8 +908,12 @@
def visitFrom(self, node):
self.set_lineno(node)
+ level = node.level
+ if level == 0 and not self.graph.checkFlag(CO_FUTURE_ABSIMPORT):
+ level = -1
fromlist = map(lambda (name, alias): name, node.names)
if VERSION > 1:
+ self.emit('LOAD_CONST', level)
self.emit('LOAD_CONST', tuple(fromlist))
self.emit('IMPORT_NAME', node.modname)
for name, alias in node.names:
@@ -909,8 +1049,6 @@
self.emit('STORE_SLICE+%d' % slice)
def visitAugSubscript(self, node, mode):
- if len(node.subs) > 1:
- raise SyntaxError, "augmented assignment to tuple is not possible"
if mode == "load":
self.visitSubscript(node, 1)
elif mode == "store":
@@ -1015,10 +1153,10 @@
self.visit(node.expr)
for sub in node.subs:
self.visit(sub)
+ if len(node.subs) > 1:
+ self.emit('BUILD_TUPLE', len(node.subs))
if aug_flag:
self.emit('DUP_TOPX', 2)
- if len(node.subs) > 1:
- self.emit('BUILD_TUPLE', len(node.subs))
if node.flags == 'OP_APPLY':
self.emit('BINARY_SUBSCR')
elif node.flags == 'OP_ASSIGN':
@@ -1204,6 +1342,7 @@
klass.lambdaCount = klass.lambdaCount + 1
else:
name = func.name
+
args, hasTupleArg = generateArgList(func.argnames)
self.graph = pyassem.PyFlowGraph(name, func.filename, args,
optimized=1)
@@ -1235,7 +1374,7 @@
def generateArgUnpack(self, args):
for i in range(len(args)):
arg = args[i]
- if type(arg) == types.TupleType:
+ if isinstance(arg, tuple):
self.emit('LOAD_FAST', '.%d' % (i * 2))
self.unpackSequence(arg)
@@ -1245,7 +1384,7 @@
else:
self.emit('UNPACK_TUPLE', len(tup))
for elt in tup:
- if type(elt) == types.TupleType:
+ if isinstance(elt, tuple):
self.unpackSequence(elt)
else:
self._nameOp('STORE', elt)
@@ -1268,6 +1407,21 @@
if self.scope.generator is not None:
self.graph.setFlag(CO_GENERATOR)
+class GenExprCodeGenerator(NestedScopeMixin, AbstractFunctionCode,
+ CodeGenerator):
+ super_init = CodeGenerator.__init__ # call be other init
+ scopes = None
+
+ __super_init = AbstractFunctionCode.__init__
+
+ def __init__(self, gexp, scopes, class_name, mod):
+ self.scopes = scopes
+ self.scope = scopes[gexp]
+ self.__super_init(gexp, scopes, 1, class_name, mod)
+ self.graph.setFreeVars(self.scope.get_free_vars())
+ self.graph.setCellVars(self.scope.get_cell_vars())
+ self.graph.setFlag(CO_GENERATOR)
+
class AbstractClassCode:
def __init__(self, klass, scopes, module):
@@ -1316,9 +1470,9 @@
count = 0
for i in range(len(arglist)):
elt = arglist[i]
- if type(elt) == types.StringType:
+ if isinstance(elt, str):
args.append(elt)
- elif type(elt) == types.TupleType:
+ elif isinstance(elt, tuple):
args.append(TupleArg(i * 2, elt))
extra.extend(misc.flatten(elt))
count = count + 1
Modified: branches/newlist/Lib/compiler/transformer.py
===================================================================
--- branches/newlist/Lib/compiler/transformer.py 2009-04-10 12:51:05 UTC (rev 6206)
+++ branches/newlist/Lib/compiler/transformer.py 2009-04-10 13:45:27 UTC (rev 6207)
@@ -14,7 +14,10 @@
#
# Modifications and improvements for Python 2.0 by Jeremy Hylton and
# Mark Hammond
-
+#
+# Some fixes to try to have correct line number on almost all nodes
+# (except Module, Discard and Stmt) added by Sylvain Thenault
+#
# Portions of this file are:
# Copyright (C) 1997-1998 Greg Stein. All Rights Reserved.
#
@@ -22,22 +25,21 @@
# http://www.opensource.org/licenses/bsd-license.html
# and replace OWNER, ORGANIZATION, and YEAR as appropriate.
-from ast import *
-#import parser
-parser = None
-# Care must be taken to use only symbols and tokens defined in Python
-# 1.5.2 for code branches executed in 1.5.2
+from compiler.ast import *
import symbol
import token
import sys
+if not sys.platform.startswith('java'):
+ import parser
-error = 'walker.error'
+class WalkerError(StandardError):
+ pass
-from consts import CO_VARARGS, CO_VARKEYWORDS
-from consts import OP_ASSIGN, OP_DELETE, OP_APPLY
+from compiler.consts import CO_VARARGS, CO_VARKEYWORDS
+from compiler.consts import OP_ASSIGN, OP_DELETE, OP_APPLY
def parseFile(path):
- f = open(path)
+ f = open(path, "U")
# XXX The parser API tolerates files without a trailing newline,
# but not strings without a trailing newline. Always add an extra
# newline to the file contents, since we're going through the string
@@ -69,6 +71,16 @@
l.append(item)
return l
+def extractLineNo(ast):
+ if not isinstance(ast[1], tuple):
+ # get a terminal node
+ return ast[2]
+ for child in ast[1:]:
+ if isinstance(child, tuple):
+ lineno = extractLineNo(child)
+ if lineno is not None:
+ return lineno
+
def Node(*args):
kind = args[0]
if nodes.has_key(kind):
@@ -78,7 +90,7 @@
print nodes[kind], len(args), args
raise
else:
- raise error, "Can't find appropriate Node type: %s" % str(args)
+ raise WalkerError, "Can't find appropriate Node type: %s" % str(args)
#return apply(ast.Node, args)
class Transformer:
@@ -109,15 +121,12 @@
def transform(self, tree):
"""Transform an AST into a modified parse tree."""
- if type(tree) != type(()) and type(tree) != type([]):
+ if not (isinstance(tree, tuple) or isinstance(tree, list)):
tree = parser.ast2tuple(tree, line_info=1)
return self.compile_node(tree)
def parsesuite(self, text):
"""Return a modified parse tree for the given suite text."""
- # Hack for handling non-native line endings on non-DOS like OSs.
- # this can go now we have universal newlines?
- text = text.replace('\x0d', '')
return self.transform(parser.suite(text))
def parseexpr(self, text):
@@ -157,7 +166,7 @@
if n == symbol.classdef:
return self.classdef(node[1:])
- raise error, ('unexpected node type', n)
+ raise WalkerError, ('unexpected node type', n)
def single_input(self, node):
### do we want to do anything about being "interactive" ?
@@ -186,31 +195,77 @@
### is this sufficient?
return Expression(self.com_node(nodelist[0]))
+ def decorator_name(self, nodelist):
+ listlen = len(nodelist)
+ assert listlen >= 1 and listlen % 2 == 1
+
+ item = self.atom_name(nodelist)
+ i = 1
+ while i < listlen:
+ assert nodelist[i][0] == token.DOT
+ assert nodelist[i + 1][0] == token.NAME
+ item = Getattr(item, nodelist[i + 1][1])
+ i += 2
+
+ return item
+
+ def decorator(self, nodelist):
+ # '@' dotted_name [ '(' [arglist] ')' ]
+ assert len(nodelist) in (3, 5, 6)
+ assert nodelist[0][0] == token.AT
+ assert nodelist[-1][0] == token.NEWLINE
+
+ assert nodelist[1][0] == symbol.dotted_name
+ funcname = self.decorator_name(nodelist[1][1:])
+
+ if len(nodelist) > 3:
+ assert nodelist[2][0] == token.LPAR
+ expr = self.com_call_function(funcname, nodelist[3])
+ else:
+ expr = funcname
+
+ return expr
+
+ def decorators(self, nodelist):
+ # decorators: decorator ([NEWLINE] decorator)* NEWLINE
+ items = []
+ for dec_nodelist in nodelist:
+ assert dec_nodelist[0] == symbol.decorator
+ items.append(self.decorator(dec_nodelist[1:]))
+ return Decorators(items)
+
def funcdef(self, nodelist):
- # funcdef: 'def' NAME parameters ':' suite
+ # -6 -5 -4 -3 -2 -1
+ # funcdef: [decorators] 'def' NAME parameters ':' suite
# parameters: '(' [varargslist] ')'
- lineno = nodelist[1][2]
- name = nodelist[1][1]
- args = nodelist[2][2]
+ if len(nodelist) == 6:
+ assert nodelist[0][0] == symbol.decorators
+ decorators = self.decorators(nodelist[0][1:])
+ else:
+ assert len(nodelist) == 5
+ decorators = None
+ lineno = nodelist[-4][2]
+ name = nodelist[-4][1]
+ args = nodelist[-3][2]
+
if args[0] == symbol.varargslist:
names, defaults, flags = self.com_arglist(args[1:])
else:
names = defaults = ()
flags = 0
- doc = self.get_docstring(nodelist[4])
+ doc = self.get_docstring(nodelist[-1])
# code for function
- code = self.com_node(nodelist[4])
+ code = self.com_node(nodelist[-1])
if doc is not None:
assert isinstance(code, Stmt)
assert isinstance(code.nodes[0], Discard)
del code.nodes[0]
- n = Function(name, names, defaults, flags, doc, code)
- n.lineno = lineno
- return n
+ return Function(decorators, name, names, defaults, flags, doc, code,
+ lineno=lineno)
def lambdef(self, nodelist):
# lambdef: 'lambda' [varargslist] ':' test
@@ -223,17 +278,18 @@
# code for lambda
code = self.com_node(nodelist[-1])
- n = Lambda(names, defaults, flags, code)
- n.lineno = nodelist[1][2]
- return n
+ return Lambda(names, defaults, flags, code, lineno=nodelist[1][2])
+ old_lambdef = lambdef
def classdef(self, nodelist):
- # classdef: 'class' NAME ['(' testlist ')'] ':' suite
+ # classdef: 'class' NAME ['(' [testlist] ')'] ':' suite
name = nodelist[1][1]
doc = self.get_docstring(nodelist[-1])
if nodelist[2][0] == token.COLON:
bases = []
+ elif nodelist[3][0] == token.RPAR:
+ bases = []
else:
bases = self.com_bases(nodelist[3])
@@ -245,9 +301,7 @@
assert isinstance(code.nodes[0], Discard)
del code.nodes[0]
- n = Class(name, bases, doc, code)
- n.lineno = nodelist[1][2]
- return n
+ return Class(name, bases, doc, code, lineno=nodelist[1][2])
def stmt(self, nodelist):
return self.com_stmt(nodelist[0])
@@ -264,31 +318,31 @@
return Stmt(stmts)
def parameters(self, nodelist):
- raise error
+ raise WalkerError
def varargslist(self, nodelist):
- raise error
+ raise WalkerError
def fpdef(self, nodelist):
- raise error
+ raise WalkerError
def fplist(self, nodelist):
- raise error
+ raise WalkerError
def dotted_name(self, nodelist):
- raise error
+ raise WalkerError
def comp_op(self, nodelist):
- raise error
+ raise WalkerError
def trailer(self, nodelist):
- raise error
+ raise WalkerError
def sliceop(self, nodelist):
- raise error
+ raise WalkerError
def argument(self, nodelist):
- raise error
+ raise WalkerError
# --------------------------------------------------------------
#
@@ -300,21 +354,17 @@
en = nodelist[-1]
exprNode = self.lookup_node(en)(en[1:])
if len(nodelist) == 1:
- n = Discard(exprNode)
- n.lineno = exprNode.lineno
- return n
+ return Discard(exprNode, lineno=exprNode.lineno)
if nodelist[1][0] == token.EQUAL:
nodesl = []
for i in range(0, len(nodelist) - 2, 2):
nodesl.append(self.com_assign(nodelist[i], OP_ASSIGN))
- n = Assign(nodesl, exprNode)
- n.lineno = nodelist[1][2]
+ return Assign(nodesl, exprNode, lineno=nodelist[1][2])
else:
lval = self.com_augassign(nodelist[0])
op = self.com_augassign_op(nodelist[1])
- n = AugAssign(lval, op[1], exprNode)
- n.lineno = op[2]
- return n
+ return AugAssign(lval, op[1], exprNode, lineno=op[2])
+ raise WalkerError, "can't get here"
def print_stmt(self, nodelist):
# print ([ test (',' test)* [','] ] | '>>' test [ (',' test)+ [','] ])
@@ -333,46 +383,38 @@
for i in range(start, len(nodelist), 2):
items.append(self.com_node(nodelist[i]))
if nodelist[-1][0] == token.COMMA:
- n = Print(items, dest)
- n.lineno = nodelist[0][2]
- return n
- n = Printnl(items, dest)
- n.lineno = nodelist[0][2]
- return n
+ return Print(items, dest, lineno=nodelist[0][2])
+ return Printnl(items, dest, lineno=nodelist[0][2])
def del_stmt(self, nodelist):
return self.com_assign(nodelist[1], OP_DELETE)
def pass_stmt(self, nodelist):
- n = Pass()
- n.lineno = nodelist[0][2]
- return n
+ return Pass(lineno=nodelist[0][2])
def break_stmt(self, nodelist):
- n = Break()
- n.lineno = nodelist[0][2]
- return n
+ return Break(lineno=nodelist[0][2])
def continue_stmt(self, nodelist):
- n = Continue()
- n.lineno = nodelist[0][2]
- return n
+ return Continue(lineno=nodelist[0][2])
def return_stmt(self, nodelist):
# return: [testlist]
if len(nodelist) < 2:
- n = Return(Const(None))
- n.lineno = nodelist[0][2]
- return n
- n = Return(self.com_node(nodelist[1]))
- n.lineno = nodelist[0][2]
- return n
+ return Return(Const(None), lineno=nodelist[0][2])
+ return Return(self.com_node(nodelist[1]), lineno=nodelist[0][2])
def yield_stmt(self, nodelist):
- n = Yield(self.com_node(nodelist[1]))
- n.lineno = nodelist[0][2]
- return n
+ expr = self.com_node(nodelist[0])
+ return Discard(expr, lineno=expr.lineno)
+ def yield_expr(self, nodelist):
+ if len(nodelist) > 1:
+ value = self.com_node(nodelist[1])
+ else:
+ value = Const(None)
+ return Yield(value, lineno=nodelist[0][2])
+
def raise_stmt(self, nodelist):
# raise: [test [',' test [',' test]]]
if len(nodelist) > 5:
@@ -387,44 +429,46 @@
expr1 = self.com_node(nodelist[1])
else:
expr1 = None
- n = Raise(expr1, expr2, expr3)
- n.lineno = nodelist[0][2]
- return n
+ return Raise(expr1, expr2, expr3, lineno=nodelist[0][2])
def import_stmt(self, nodelist):
- # import_stmt: 'import' dotted_as_name (',' dotted_as_name)* |
- # from: 'from' dotted_name 'import'
- # ('*' | import_as_name (',' import_as_name)*)
- if nodelist[0][1] == 'from':
- names = []
- if nodelist[3][0] == token.NAME:
- for i in range(3, len(nodelist), 2):
- names.append((nodelist[i][1], None))
- else:
- for i in range(3, len(nodelist), 2):
- names.append(self.com_import_as_name(nodelist[i]))
- n = From(self.com_dotted_name(nodelist[1]), names)
- n.lineno = nodelist[0][2]
- return n
+ # import_stmt: import_name | import_from
+ assert len(nodelist) == 1
+ return self.com_node(nodelist[0])
- if nodelist[1][0] == symbol.dotted_name:
- names = [(self.com_dotted_name(nodelist[1][1:]), None)]
+ def import_name(self, nodelist):
+ # import_name: 'import' dotted_as_names
+ return Import(self.com_dotted_as_names(nodelist[1]),
+ lineno=nodelist[0][2])
+
+ def import_from(self, nodelist):
+ # import_from: 'from' ('.'* dotted_name | '.') 'import' ('*' |
+ # '(' import_as_names ')' | import_as_names)
+ assert nodelist[0][1] == 'from'
+ idx = 1
+ while nodelist[idx][1] == '.':
+ idx += 1
+ level = idx - 1
+ if nodelist[idx][0] == symbol.dotted_name:
+ fromname = self.com_dotted_name(nodelist[idx])
+ idx += 1
else:
- names = []
- for i in range(1, len(nodelist), 2):
- names.append(self.com_dotted_as_name(nodelist[i]))
- n = Import(names)
- n.lineno = nodelist[0][2]
- return n
+ fromname = ""
+ assert nodelist[idx][1] == 'import'
+ if nodelist[idx + 1][0] == token.STAR:
+ return From(fromname, [('*', None)], level,
+ lineno=nodelist[0][2])
+ else:
+ node = nodelist[idx + 1 + (nodelist[idx + 1][0] == token.LPAR)]
+ return From(fromname, self.com_import_as_names(node), level,
+ lineno=nodelist[0][2])
def global_stmt(self, nodelist):
# global: NAME (',' NAME)*
names = []
for i in range(1, len(nodelist), 2):
names.append(nodelist[i][1])
- n = Global(names)
- n.lineno = nodelist[0][2]
- return n
+ return Global(names, lineno=nodelist[0][2])
def exec_stmt(self, nodelist):
# exec_stmt: 'exec' expr ['in' expr [',' expr]]
@@ -438,9 +482,7 @@
else:
expr2 = expr3 = None
- n = Exec(expr1, expr2, expr3)
- n.lineno = nodelist[0][2]
- return n
+ return Exec(expr1, expr2, expr3, lineno=nodelist[0][2])
def assert_stmt(self, nodelist):
# 'assert': test, [',' test]
@@ -449,9 +491,7 @@
expr2 = self.com_node(nodelist[3])
else:
expr2 = None
- n = Assert(expr1, expr2)
- n.lineno = nodelist[0][2]
- return n
+ return Assert(expr1, expr2, lineno=nodelist[0][2])
def if_stmt(self, nodelist):
# if: test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
@@ -466,9 +506,7 @@
## elseNode.lineno = nodelist[-1][1][2]
else:
elseNode = None
- n = If(tests, elseNode)
- n.lineno = nodelist[0][2]
- return n
+ return If(tests, elseNode, lineno=nodelist[0][2])
def while_stmt(self, nodelist):
# 'while' test ':' suite ['else' ':' suite]
@@ -481,9 +519,7 @@
else:
elseNode = None
- n = While(testNode, bodyNode, elseNode)
- n.lineno = nodelist[0][2]
- return n
+ return While(testNode, bodyNode, elseNode, lineno=nodelist[0][2])
def for_stmt(self, nodelist):
# 'for' exprlist 'in' exprlist ':' suite ['else' ':' suite]
@@ -497,18 +533,18 @@
else:
elseNode = None
- n = For(assignNode, listNode, bodyNode, elseNode)
- n.lineno = nodelist[0][2]
- return n
+ return For(assignNode, listNode, bodyNode, elseNode,
+ lineno=nodelist[0][2])
def try_stmt(self, nodelist):
- # 'try' ':' suite (except_clause ':' suite)+ ['else' ':' suite]
- # | 'try' ':' suite 'finally' ':' suite
- if nodelist[3][0] != symbol.except_clause:
- return self.com_try_finally(nodelist)
+ return self.com_try_except_finally(nodelist)
- return self.com_try_except(nodelist)
+ def with_stmt(self, nodelist):
+ return self.com_with(nodelist)
+ def with_var(self, nodelist):
+ return self.com_with_var(nodelist)
+
def suite(self, nodelist):
# simple_stmt | NEWLINE INDENT NEWLINE* (stmt NEWLINE*)+ DEDENT
if len(nodelist) == 1:
@@ -535,11 +571,32 @@
testlist1 = testlist
exprlist = testlist
+ def testlist_gexp(self, nodelist):
+ if len(nodelist) == 2 and nodelist[1][0] == symbol.gen_for:
+ test = self.com_node(nodelist[0])
+ return self.com_generator_expression(test, nodelist[1])
+ return self.testlist(nodelist)
+
def test(self, nodelist):
+ # or_test ['if' or_test 'else' test] | lambdef
+ if len(nodelist) == 1 and nodelist[0][0] == symbol.lambdef:
+ return self.lambdef(nodelist[0])
+ then = self.com_node(nodelist[0])
+ if len(nodelist) > 1:
+ assert len(nodelist) == 5
+ assert nodelist[1][1] == 'if'
+ assert nodelist[3][1] == 'else'
+ test = self.com_node(nodelist[2])
+ else_ = self.com_node(nodelist[4])
+ return IfExp(test, then, else_, lineno=nodelist[1][2])
+ return then
+
+ def or_test(self, nodelist):
# and_test ('or' and_test)* | lambdef
if len(nodelist) == 1 and nodelist[0][0] == symbol.lambdef:
return self.lambdef(nodelist[0])
return self.com_binary(Or, nodelist)
+ old_test = or_test
def and_test(self, nodelist):
# not_test ('and' not_test)*
@@ -549,9 +606,7 @@
# 'not' not_test | comparison
result = self.com_node(nodelist[-1])
if len(nodelist) == 2:
- n = Not(result)
- n.lineno = nodelist[0][2]
- return n
+ return Not(result, lineno=nodelist[0][2])
return result
def comparison(self, nodelist):
@@ -585,9 +640,7 @@
# the two have very different semantics and results (note that the
# latter form is always true)
- n = Compare(node, results)
- n.lineno = lineno
- return n
+ return Compare(node, results, lineno=lineno)
def expr(self, nodelist):
# xor_expr ('|' xor_expr)*
@@ -607,11 +660,9 @@
for i in range(2, len(nodelist), 2):
right = self.com_node(nodelist[i])
if nodelist[i-1][0] == token.LEFTSHIFT:
- node = LeftShift([node, right])
- node.lineno = nodelist[1][2]
+ node = LeftShift([node, right], lineno=nodelist[1][2])
elif nodelist[i-1][0] == token.RIGHTSHIFT:
- node = RightShift([node, right])
- node.lineno = nodelist[1][2]
+ node = RightShift([node, right], lineno=nodelist[1][2])
else:
raise ValueError, "unexpected token: %s" % nodelist[i-1][0]
return node
@@ -621,11 +672,9 @@
for i in range(2, len(nodelist), 2):
right = self.com_node(nodelist[i])
if nodelist[i-1][0] == token.PLUS:
- node = Add([node, right])
- node.lineno = nodelist[1][2]
+ node = Add([node, right], lineno=nodelist[1][2])
elif nodelist[i-1][0] == token.MINUS:
- node = Sub([node, right])
- node.lineno = nodelist[1][2]
+ node = Sub([node, right], lineno=nodelist[1][2])
else:
raise ValueError, "unexpected token: %s" % nodelist[i-1][0]
return node
@@ -651,17 +700,14 @@
def factor(self, nodelist):
elt = nodelist[0]
t = elt[0]
- node = self.com_node(nodelist[-1])
+ node = self.lookup_node(nodelist[-1])(nodelist[-1][1:])
# need to handle (unary op)constant here...
if t == token.PLUS:
- node = UnaryAdd(node)
- node.lineno = elt[2]
+ return UnaryAdd(node, lineno=elt[2])
elif t == token.MINUS:
- node = UnarySub(node)
- node.lineno = elt[2]
+ return UnarySub(node, lineno=elt[2])
elif t == token.TILDE:
- node = Invert(node)
- node.lineno = elt[2]
+ node = Invert(node, lineno=elt[2])
return node
def power(self, nodelist):
@@ -670,49 +716,38 @@
for i in range(1, len(nodelist)):
elt = nodelist[i]
if elt[0] == token.DOUBLESTAR:
- n = Power([node, self.com_node(nodelist[i+1])])
- n.lineno = elt[2]
- return n
+ return Power([node, self.com_node(nodelist[i+1])],
+ lineno=elt[2])
node = self.com_apply_trailer(node, elt)
return node
def atom(self, nodelist):
- n = self._atom_dispatch[nodelist[0][0]](nodelist)
- n.lineno = nodelist[0][2]
- return n
+ return self._atom_dispatch[nodelist[0][0]](nodelist)
def atom_lpar(self, nodelist):
if nodelist[1][0] == token.RPAR:
- n = Tuple(())
- n.lineno = nodelist[0][2]
- return n
+ return Tuple((), lineno=nodelist[0][2])
return self.com_node(nodelist[1])
def atom_lsqb(self, nodelist):
if nodelist[1][0] == token.RSQB:
- n = List(())
- n.lineno = nodelist[0][2]
- return n
+ return List((), lineno=nodelist[0][2])
return self.com_list_constructor(nodelist[1])
def atom_lbrace(self, nodelist):
if nodelist[1][0] == token.RBRACE:
- return Dict(())
+ return Dict((), lineno=nodelist[0][2])
return self.com_dictmaker(nodelist[1])
def atom_backquote(self, nodelist):
- n = Backquote(self.com_node(nodelist[1]))
- n.lineno = nodelist[0][2]
- return n
+ return Backquote(self.com_node(nodelist[1]))
def atom_number(self, nodelist):
### need to verify this matches compile.c
k = eval(nodelist[0][1])
- n = Const(k)
- n.lineno = nodelist[0][2]
- return n
+ return Const(k, lineno=nodelist[0][2])
def decode_literal(self, lit):
if self.encoding:
@@ -729,15 +764,10 @@
k = ''
for node in nodelist:
k += self.decode_literal(node[1])
- n = Const(k)
- n.lineno = nodelist[0][2]
- return n
+ return Const(k, lineno=nodelist[0][2])
def atom_name(self, nodelist):
- ### any processing to do?
- n = Name(nodelist[0][1])
- n.lineno = nodelist[0][2]
- return n
+ return Name(nodelist[0][1], lineno=nodelist[0][2])
# --------------------------------------------------------------
#
@@ -805,16 +835,15 @@
names.append(self.com_fpdef(node))
i = i + 1
- if i >= len(nodelist):
- break
-
- if nodelist[i][0] == token.EQUAL:
+ if i < len(nodelist) and nodelist[i][0] == token.EQUAL:
defaults.append(self.com_node(nodelist[i + 1]))
i = i + 2
elif len(defaults):
- # Treat "(a=1, b)" as "(a=1, b=None)"
- defaults.append(Const(None))
+ # we have already seen an argument with default, but here
+ # came one without
+ raise SyntaxError, "non-default argument follows default argument"
+ # skip the comma
i = i + 1
return names, defaults, flags
@@ -843,48 +872,62 @@
return name[:-1]
def com_dotted_as_name(self, node):
- dot = self.com_dotted_name(node[1])
- if len(node) <= 2:
+ assert node[0] == symbol.dotted_as_name
+ node = node[1:]
+ dot = self.com_dotted_name(node[0][1:])
+ if len(node) == 1:
return dot, None
- if node[0] == symbol.dotted_name:
- pass
- else:
- assert node[2][1] == 'as'
- assert node[3][0] == token.NAME
- return dot, node[3][1]
+ assert node[1][1] == 'as'
+ assert node[2][0] == token.NAME
+ return dot, node[2][1]
+ def com_dotted_as_names(self, node):
+ assert node[0] == symbol.dotted_as_names
+ node = node[1:]
+ names = [self.com_dotted_as_name(node[0])]
+ for i in range(2, len(node), 2):
+ names.append(self.com_dotted_as_name(node[i]))
+ return names
+
def com_import_as_name(self, node):
- if node[0] == token.STAR:
- return '*', None
assert node[0] == symbol.import_as_name
node = node[1:]
+ assert node[0][0] == token.NAME
if len(node) == 1:
- assert node[0][0] == token.NAME
return node[0][1], None
-
assert node[1][1] == 'as', node
assert node[2][0] == token.NAME
return node[0][1], node[2][1]
+ def com_import_as_names(self, node):
+ assert node[0] == symbol.import_as_names
+ node = node[1:]
+ names = [self.com_import_as_name(node[0])]
+ for i in range(2, len(node), 2):
+ names.append(self.com_import_as_name(node[i]))
+ return names
+
def com_bases(self, node):
bases = []
for i in range(1, len(node), 2):
bases.append(self.com_node(node[i]))
return bases
- def com_try_finally(self, nodelist):
- # try_fin_stmt: "try" ":" suite "finally" ":" suite
- n = TryFinally(self.com_node(nodelist[2]),
- self.com_node(nodelist[5]))
- n.lineno = nodelist[0][2]
- return n
+ def com_try_except_finally(self, nodelist):
+ # ('try' ':' suite
+ # ((except_clause ':' suite)+ ['else' ':' suite] ['finally' ':' suite]
+ # | 'finally' ':' suite))
- def com_try_except(self, nodelist):
- # try_except: 'try' ':' suite (except_clause ':' suite)* ['else' suite]
+ if nodelist[3][0] == token.NAME:
+ # first clause is a finally clause: only try-finally
+ return TryFinally(self.com_node(nodelist[2]),
+ self.com_node(nodelist[5]),
+ lineno=nodelist[0][2])
+
#tryexcept: [TryNode, [except_clauses], elseNode)]
- stmt = self.com_node(nodelist[2])
clauses = []
elseNode = None
+ finallyNode = None
for i in range(3, len(nodelist), 3):
node = nodelist[i]
if node[0] == symbol.except_clause:
@@ -900,11 +943,31 @@
clauses.append((expr1, expr2, self.com_node(nodelist[i+2])))
if node[0] == token.NAME:
- elseNode = self.com_node(nodelist[i+2])
- n = TryExcept(self.com_node(nodelist[2]), clauses, elseNode)
- n.lineno = nodelist[0][2]
- return n
+ if node[1] == 'else':
+ elseNode = self.com_node(nodelist[i+2])
+ elif node[1] == 'finally':
+ finallyNode = self.com_node(nodelist[i+2])
+ try_except = TryExcept(self.com_node(nodelist[2]), clauses, elseNode,
+ lineno=nodelist[0][2])
+ if finallyNode:
+ return TryFinally(try_except, finallyNode, lineno=nodelist[0][2])
+ else:
+ return try_except
+ def com_with(self, nodelist):
+ # with_stmt: 'with' expr [with_var] ':' suite
+ expr = self.com_node(nodelist[1])
+ body = self.com_node(nodelist[-1])
+ if nodelist[2][0] == token.COLON:
+ var = None
+ else:
+ var = self.com_assign(nodelist[2][2], OP_ASSIGN)
+ return With(expr, var, body, lineno=nodelist[0][2])
+
+ def com_with_var(self, nodelist):
+ # with_var: 'as' expr
+ return self.com_node(nodelist[1])
+
def com_augassign_op(self, node):
assert node[0] == symbol.augassign
return node[1]
@@ -924,7 +987,7 @@
# loop to avoid trivial recursion
while 1:
t = node[0]
- if t == symbol.exprlist or t == symbol.testlist:
+ if t in (symbol.exprlist, symbol.testlist, symbol.testlist_safe, symbol.testlist_gexp):
if len(node) > 2:
return self.com_assign_tuple(node, assigning)
node = node[1]
@@ -961,13 +1024,13 @@
else:
raise SyntaxError, "can't assign to literal"
else:
- raise SyntaxError, "bad assignment"
+ raise SyntaxError, "bad assignment (%s)" % t
def com_assign_tuple(self, node, assigning):
assigns = []
for i in range(1, len(node), 2):
assigns.append(self.com_assign(node[i], assigning))
- return AssTuple(assigns)
+ return AssTuple(assigns, lineno=extractLineNo(node))
def com_assign_list(self, node, assigning):
assigns = []
@@ -977,12 +1040,10 @@
raise SyntaxError, "can't assign to list comprehension"
assert node[i + 1][0] == token.COMMA, node[i + 1]
assigns.append(self.com_assign(node[i], assigning))
- return AssList(assigns)
+ return AssList(assigns, lineno=extractLineNo(node))
def com_assign_name(self, node, assigning):
- n = AssName(node[1], assigning)
- n.lineno = node[2]
- return n
+ return AssName(node[1], assigning, lineno=node[2])
def com_assign_trailer(self, primary, node, assigning):
t = node[1][0]
@@ -995,7 +1056,7 @@
raise SyntaxError, "unknown trailer type: %s" % t
def com_assign_attr(self, primary, node, assigning):
- return AssAttr(primary, node[1], assigning)
+ return AssAttr(primary, node[1], assigning, lineno=node[-1])
def com_binary(self, constructor, nodelist):
"Compile 'NODE (OP NODE)*' into (type, [ node1, ..., nodeN ])."
@@ -1007,7 +1068,7 @@
for i in range(0, l, 2):
n = nodelist[i]
items.append(self.lookup_node(n)(n[1:]))
- return constructor(items)
+ return constructor(items, lineno=extractLineNo(nodelist))
def com_stmt(self, node):
result = self.lookup_node(node)(node[1:])
@@ -1017,7 +1078,7 @@
return Stmt([result])
def com_append_stmt(self, stmts, node):
- result = self.com_node(node)
+ result = self.lookup_node(node)(node[1:])
assert result is not None
if isinstance(result, Stmt):
stmts.extend(result.nodes)
@@ -1036,7 +1097,7 @@
elif nodelist[i][0] == token.COMMA:
continue
values.append(self.com_node(nodelist[i]))
- return List(values)
+ return List(values, lineno=values[0].lineno)
def com_list_comprehension(self, expr, node):
# list_iter: list_for | list_if
@@ -1061,8 +1122,7 @@
node = self.com_list_iter(node[5])
elif t == 'if':
test = self.com_node(node[2])
- newif = ListCompIf(test)
- newif.lineno = node[1][2]
+ newif = ListCompIf(test, lineno=node[1][2])
newfor.ifs.append(newif)
if len(node) == 3:
node = None
@@ -1072,9 +1132,7 @@
raise SyntaxError, \
("unexpected list comprehension element: %s %d"
% (node, lineno))
- n = ListComp(expr, fors)
- n.lineno = lineno
- return n
+ return ListComp(expr, fors, lineno=lineno)
def com_list_iter(self, node):
assert node[0] == symbol.list_iter
@@ -1084,15 +1142,54 @@
values = []
for i in range(1, len(nodelist), 2):
values.append(self.com_node(nodelist[i]))
- return List(values)
+ return List(values, lineno=values[0].lineno)
+ if hasattr(symbol, 'gen_for'):
+ def com_generator_expression(self, expr, node):
+ # gen_iter: gen_for | gen_if
+ # gen_for: 'for' exprlist 'in' test [gen_iter]
+ # gen_if: 'if' test [gen_iter]
+
+ lineno = node[1][2]
+ fors = []
+ while node:
+ t = node[1][1]
+ if t == 'for':
+ assignNode = self.com_assign(node[2], OP_ASSIGN)
+ genNode = self.com_node(node[4])
+ newfor = GenExprFor(assignNode, genNode, [],
+ lineno=node[1][2])
+ fors.append(newfor)
+ if (len(node)) == 5:
+ node = None
+ else:
+ node = self.com_gen_iter(node[5])
+ elif t == 'if':
+ test = self.com_node(node[2])
+ newif = GenExprIf(test, lineno=node[1][2])
+ newfor.ifs.append(newif)
+ if len(node) == 3:
+ node = None
+ else:
+ node = self.com_gen_iter(node[3])
+ else:
+ raise SyntaxError, \
+ ("unexpected generator expression element: %s %d"
+ % (node, lineno))
+ fors[0].is_outmost = True
+ return GenExpr(GenExprInner(expr, fors), lineno=lineno)
+
+ def com_gen_iter(self, node):
+ assert node[0] == symbol.gen_...
[truncated message content] |
