Menu
▾
▴
[Jython-checkins] SF.net SVN: jython:[6046] trunk/jython
|
From: <zy...@us...> - 2009-02-24 05:45:43
|
Revision: 6046
http://jython.svn.sourceforge.net/jython/?rev=6046&view=rev
Author: zyasoft
Date: 2009-02-24 05:45:38 +0000 (Tue, 24 Feb 2009)
Log Message:
-----------
Merged revisions 5949,5952,5962,5964,5967-5970,5980,5984,5987-5989,5998-5999,6004-6005,6015-6022,6024-6030,6038,6041-6042 via svnmerge from
https://jython.svn.sourceforge.net/svnroot/jython/branches/pbcvm
........
r5949 | zyasoft | 2009-01-20 06:56:46 -0700 (Tue, 20 Jan 2009) | 2 lines
Initial commit.
........
r5952 | zyasoft | 2009-01-20 22:02:29 -0700 (Tue, 20 Jan 2009) | 3 lines
Fixed binary and inplace bytecodes to use the proper methods.
Added block support to support iteration.
........
r5962 | zyasoft | 2009-01-22 21:40:26 -0700 (Thu, 22 Jan 2009) | 3 lines
Added support for try/except and try/finally blocks (and their
combination).
........
r5964 | zyasoft | 2009-01-23 08:30:09 -0700 (Fri, 23 Jan 2009) | 2 lines
Basic generators are almost supported.
........
r5967 | zyasoft | 2009-01-23 19:49:52 -0700 (Fri, 23 Jan 2009) | 2 lines
Now supports generators, via the standard PyGenerator protocol.
........
r5968 | zyasoft | 2009-01-23 21:36:06 -0700 (Fri, 23 Jan 2009) | 2 lines
Added support for keyword args, *arg, and **kwarg calling conventions.
........
r5969 | zyasoft | 2009-01-24 09:47:59 -0700 (Sat, 24 Jan 2009) | 3 lines
Added support for basic coroutines (no support for throw) and
generator expressions.
........
r5970 | zyasoft | 2009-01-24 11:02:37 -0700 (Sat, 24 Jan 2009) | 2 lines
Supports throwing exceptions into generators.
........
r5980 | zyasoft | 2009-01-25 18:19:12 -0700 (Sun, 25 Jan 2009) | 4 lines
Properly handles *args, **kwargs for the receiver function.
func_code can now be set to a PyBytecode (or other extender of PyBaseCode).
Turned off debugging by default for PyBytecode objects.
........
r5984 | zyasoft | 2009-01-25 22:02:21 -0700 (Sun, 25 Jan 2009) | 2 lines
Added extract tool (to be run by CPython) for creating test modules.
........
r5987 | zyasoft | 2009-01-26 18:45:48 -0700 (Mon, 26 Jan 2009) | 2 lines
Fixed raising of exceptions and PyStack.rot
........
r5988 | zyasoft | 2009-01-26 19:16:49 -0700 (Mon, 26 Jan 2009) | 2 lines
Fixed opcode support for LOAD/STORE/DELETE_NAME and BUILD_CLASS.
........
r5989 | zyasoft | 2009-01-26 19:26:20 -0700 (Mon, 26 Jan 2009) | 2 lines
Fixed RAISE_VARARGS opcode to raise exceptions properly.
........
r5998 | zyasoft | 2009-01-28 20:44:58 -0700 (Wed, 28 Jan 2009) | 3 lines
extract.py now creates modules where codeobjects respect their
dependency relationships (via a tsort).
........
r5999 | zyasoft | 2009-01-29 07:24:33 -0700 (Thu, 29 Jan 2009) | 6 lines
Extract script now processes most/all of the test.test_* modules with
a test_main and importable by CPython. May want to consider
generalizing support for instance methods and closure setup with
respect to new.function (= PyFunction), but this seems purely a
testing nicety for top-level defs.
........
r6004 | zyasoft | 2009-02-01 01:12:13 -0700 (Sun, 01 Feb 2009) | 3 lines
From
http://svn.python.org/projects/python/branches/release25-maint/Lib/test/test_marshal@27825
........
r6005 | zyasoft | 2009-02-01 01:24:06 -0700 (Sun, 01 Feb 2009) | 13 lines
Added _marshal, a Java implementation of the Python module, and made
current. test_marshal now passes (with minor modifications around GC
and floating point accuracy on to/from string representation), except
for support of code objects. That's next.
Pulled IOFile (now PyIOFile) from cPickle to share its duck typing
with _marshal
A historical note: from what I can discern in the log, the original
marshal.py was written by Guido van Rossum specifically for "JPython"
in 1997, one of the few bits he apparently did. Now it survives only
as a shell, to import in _marshal.
........
r6015 | zyasoft | 2009-02-05 20:50:10 -0700 (Thu, 05 Feb 2009) | 5 lines
Changed read/write binary_float so that it doesn't go through struct
but instead j.l.Double's support for converting to/from bits. This
also resolves endian ordering issues so that it follows the native
endian ordering.
........
r6016 | zyasoft | 2009-02-06 00:07:45 -0700 (Fri, 06 Feb 2009) | 7 lines
Fixed some cellvar/freevar nested scope issues, with work to continue.
Added a slightly modified version of pycimport.py from Tobias' pyasm
implementation in the sandbox. This, in conjunction with compileall
(run from CPython; compileall.compile_dir against the unit tests) is
allowing a significant fraction of the regrtest to run against PBC-VM.
........
r6017 | zyasoft | 2009-02-06 00:42:15 -0700 (Fri, 06 Feb 2009) | 2 lines
Fixed slice, exec support.
........
r6018 | zyasoft | 2009-02-06 21:04:54 -0700 (Fri, 06 Feb 2009) | 2 lines
Fixed stack ops (DUP_TOPX, ROT_THREE, ROT_FOUR), INPLACE_POWER.
........
r6019 | zyasoft | 2009-02-06 23:29:37 -0700 (Fri, 06 Feb 2009) | 5 lines
Fixed IMPORT_STAR for nested modules. sys.exc_info now returns Py.None
for the traceback (this fixes an issue in PBC where null is being used
as both a sentinel value by __getitem__ AND an actual value for
exception handling).
........
r6020 | zyasoft | 2009-02-07 10:47:19 -0700 (Sat, 07 Feb 2009) | 2 lines
Fixes nested scope treatment of free variables.
........
r6021 | zyasoft | 2009-02-07 11:14:33 -0700 (Sat, 07 Feb 2009) | 2 lines
Further refinement of nested scopes.
........
r6022 | zyasoft | 2009-02-07 14:26:31 -0700 (Sat, 07 Feb 2009) | 3 lines
Fixed control flow by pushing the wrapped Why onto the stack (instead
of its string representation from the earliest version of this code).
........
r6024 | zyasoft | 2009-02-10 20:11:10 -0700 (Tue, 10 Feb 2009) | 7 lines
Support tostring/fromstring of unsigned types so that they can be
stored in the same size as their signed representation.
This fixes an issues with peak.util.assembler where it expects that
unsigned bytes (typecode 'B') take one byte to store and therefore are
directly interoperable with co_code.
........
r6025 | zyasoft | 2009-02-11 21:38:04 -0700 (Wed, 11 Feb 2009) | 2 lines
Beginning of PBC compilation support. Pretty much a shell for now.
........
r6026 | zyasoft | 2009-02-14 02:23:58 -0700 (Sat, 14 Feb 2009) | 7 lines
Now interns names in co_names, etc., when unmarshalling, which fixes
equality tests on special case like __class__, __dict__, etc.
Fix instanceof test in eval and exec so that it looks at PyBaseCode,
not PyTableCode. Also, exec cannot be called with Py.None instead of
null. Fixes test_scope.
........
r6027 | zyasoft | 2009-02-14 03:44:17 -0700 (Sat, 14 Feb 2009) | 3 lines
Set the traceback when intercepting an exception.
Set __doc__ strings for functions from the constant pool (if available).
........
r6028 | zyasoft | 2009-02-14 04:17:17 -0700 (Sat, 14 Feb 2009) | 3 lines
LOAD_DEREF needs to do the same lookup as LOAD_CLOSURE.
Populate doc strings for MAKE_CLOSURE.
........
r6029 | zyasoft | 2009-02-14 04:58:48 -0700 (Sat, 14 Feb 2009) | 3 lines
Fixed WITH_CLEANUP so that it properly calls __exit__ with the
exception components off the stack.
........
r6030 | zyasoft | 2009-02-14 12:17:04 -0700 (Sat, 14 Feb 2009) | 3 lines
Fixed UNPACK_SEQUENCE so that ValueError is raised if too few/too many
values to unpack.
........
r6038 | zyasoft | 2009-02-17 17:06:55 -0700 (Tue, 17 Feb 2009) | 8 lines
Added support for code.getlineno (using co_lnotab), co_code,
co_consts, co_lnotab. co_code and co_lnotab now use byte[] for a more
compact representation and then manage accessing as if it were an
unsigned representation.
This enables a good chunk of the peak.util.assembler unit tests to
pass; the remainder seem to have a dictionary traversal dependency.
........
r6041 | zyasoft | 2009-02-22 07:40:49 -0700 (Sun, 22 Feb 2009) | 4 lines
Inlined PyTableCode#interpret into #call(PyFrame,PyClosure) (which
restores this method to what it was before the PBC-VM work). This
change restores the performance as tested by test/pystone.py.
........
r6042 | zyasoft | 2009-02-23 21:25:13 -0700 (Mon, 23 Feb 2009) | 8 lines
LineCache implementation to support efficient setline edge detection,
as necessary for tracing.
No longer looks up the ThreadState twice, in the caller of
PyBytecode#interpret and interpret itself, so this should provide for
some optimization.
........
Modified Paths:
--------------
trunk/jython/Lib/marshal.py
trunk/jython/Lib/new.py
trunk/jython/Lib/test/test_array.py
trunk/jython/src/org/python/core/Py.java
trunk/jython/src/org/python/core/PyArray.java
trunk/jython/src/org/python/core/PyFrame.java
trunk/jython/src/org/python/core/PyFunction.java
trunk/jython/src/org/python/core/PySystemState.java
trunk/jython/src/org/python/core/PyTableCode.java
trunk/jython/src/org/python/core/__builtin__.java
trunk/jython/src/org/python/core/imp.java
trunk/jython/src/org/python/modules/Setup.java
trunk/jython/src/org/python/modules/cPickle.java
trunk/jython/src/org/python/modules/struct.java
Added Paths:
-----------
trunk/jython/Lib/pycimport.py
trunk/jython/Lib/test/test_marshal.py
trunk/jython/Tools/pbcvm/
trunk/jython/Tools/pbcvm/extract.py
trunk/jython/src/org/python/compiler/pbc/
trunk/jython/src/org/python/compiler/pbc/Bytecode.java
trunk/jython/src/org/python/compiler/pbc/BytecodeCompiler.java
trunk/jython/src/org/python/core/Opcode.java
trunk/jython/src/org/python/core/PyBaseCode.java
trunk/jython/src/org/python/core/PyBytecode.java
trunk/jython/src/org/python/modules/PyIOFile.java
trunk/jython/src/org/python/modules/PyIOFileFactory.java
trunk/jython/src/org/python/modules/_marshal.java
Removed Paths:
-------------
trunk/jython/Tools/pbcvm/extract.py
trunk/jython/src/org/python/compiler/pbc/Bytecode.java
trunk/jython/src/org/python/compiler/pbc/BytecodeCompiler.java
Property Changed:
----------------
trunk/jython/
Property changes on: trunk/jython
___________________________________________________________________
Modified: svnmerge-integrated
- /branches/pbcvm:1-5948
+ /branches/pbcvm:1-6045
Modified: trunk/jython/Lib/marshal.py
===================================================================
--- trunk/jython/Lib/marshal.py 2009-02-24 05:04:05 UTC (rev 6045)
+++ trunk/jython/Lib/marshal.py 2009-02-24 05:45:38 UTC (rev 6046)
@@ -5,332 +5,24 @@
"""
-import StringIO
-import string
-from types import *
-try:
- import new
-except ImportError:
- new = None
+import cStringIO
+from _marshal import Marshaller, Unmarshaller
-TYPE_NULL = '0'
-TYPE_NONE = 'N'
-TYPE_FALSE = 'F'
-TYPE_TRUE = 'T'
-TYPE_ELLIPSIS = '.'
-TYPE_INT = 'i'
-TYPE_INT64 = 'I'
-TYPE_FLOAT = 'f'
-TYPE_COMPLEX = 'x'
-TYPE_LONG = 'l'
-TYPE_STRING = 's'
-TYPE_TUPLE = '('
-TYPE_LIST = '['
-TYPE_DICT = '{'
-TYPE_CODE = 'c'
-TYPE_UNKNOWN = '?'
+def dump(x, f, version=2):
+ Marshaller(f, version).dump(x)
+# XXX - added just for debugging. remove!
+def load(f, debug=False):
+ u = Unmarshaller(f)
+ if debug:
+ u._debug()
+ return u.load()
-class Marshaller:
-
- dispatch = {}
-
- def __init__(self, f):
- self.f = f
-
- def dump(self, x):
- self.dispatch[type(x)](self, x)
-
- def w_long64(self, x):
- self.w_long(x)
- self.w_long(x>>32)
-
- def w_long(self, x):
- write = self.f.write
- write(chr((x) & 0xff))
- write(chr((x>> 8) & 0xff))
- write(chr((x>>16) & 0xff))
- write(chr((x>>24) & 0xff))
-
- def w_short(self, x):
- write = self.f.write
- write(chr((x) & 0xff))
- write(chr((x>> 8) & 0xff))
-
- def dump_none(self, x):
- self.f.write(TYPE_NONE)
- dispatch[NoneType] = dump_none
-
- def dump_bool(self, x):
- if x:
- self.f.write(TYPE_TRUE)
- else:
- self.f.write(TYPE_FALSE)
- dispatch[BooleanType] = dump_bool
-
- def dump_ellipsis(self, x):
- self.f.write(TYPE_ELLIPSIS)
- try:
- dispatch[EllipsisType] = dump_ellipsis
- except NameError:
- pass
-
- def dump_int(self, x):
- y = x>>31
- if y and y != -1:
- self.f.write(TYPE_INT64)
- self.w_long64(x)
- else:
- self.f.write(TYPE_INT)
- self.w_long(x)
- dispatch[IntType] = dump_int
-
- def dump_long(self, x):
- self.f.write(TYPE_LONG)
- sign = 1
- if x < 0:
- sign = -1
- x = -x
- digits = []
- while x:
- digits.append(x & 0x7FFF)
- x = x>>15
- self.w_long(len(digits) * sign)
- for d in digits:
- self.w_short(d)
- dispatch[LongType] = dump_long
-
- def dump_float(self, x):
- write = self.f.write
- write(TYPE_FLOAT)
- s = `x`
- write(chr(len(s)))
- write(s)
- dispatch[FloatType] = dump_float
-
- def dump_complex(self, x):
- write = self.f.write
- write(TYPE_COMPLEX)
- s = `x.real`
- write(chr(len(s)))
- write(s)
- s = `x.imag`
- write(chr(len(s)))
- write(s)
- try:
- dispatch[ComplexType] = dump_complex
- except NameError:
- pass
-
- def dump_string(self, x):
- self.f.write(TYPE_STRING)
- self.w_long(len(x))
- self.f.write(x)
- dispatch[StringType] = dump_string
-
- def dump_tuple(self, x):
- self.f.write(TYPE_TUPLE)
- self.w_long(len(x))
- for item in x:
- self.dump(item)
- dispatch[TupleType] = dump_tuple
-
- def dump_list(self, x):
- self.f.write(TYPE_LIST)
- self.w_long(len(x))
- for item in x:
- self.dump(item)
- dispatch[ListType] = dump_list
-
- def dump_dict(self, x):
- self.f.write(TYPE_DICT)
- for key, value in x.items():
- self.dump(key)
- self.dump(value)
- self.f.write(TYPE_NULL)
- dispatch[DictionaryType] = dump_dict
-
- def dump_code(self, x):
- self.f.write(TYPE_CODE)
- self.w_short(x.co_argcount)
- self.w_short(x.co_nlocals)
- self.w_short(x.co_stacksize)
- self.w_short(x.co_flags)
- self.dump(x.co_code)
- self.dump(x.co_consts)
- self.dump(x.co_names)
- self.dump(x.co_varnames)
- self.dump(x.co_filename)
- self.dump(x.co_name)
- self.w_short(x.co_firstlineno)
- self.dump(x.co_lnotab)
- try:
- dispatch[CodeType] = dump_code
- except NameError:
- pass
-
-
-class NULL:
- pass
-
-class Unmarshaller:
-
- dispatch = {}
-
- def __init__(self, f):
- self.f = f
-
- def load(self):
- c = self.f.read(1)
- if not c:
- raise EOFError
- return self.dispatch[c](self)
-
- def r_short(self):
- read = self.f.read
- lo = ord(read(1))
- hi = ord(read(1))
- x = lo | (hi<<8)
- if x & 0x8000:
- x = x - 0x10000
- return x
-
- def r_long(self):
- read = self.f.read
- a = ord(read(1))
- b = ord(read(1))
- c = ord(read(1))
- d = ord(read(1))
- # convert unsigned d to signed to avoid d << 24 possibly
- # overflowing into a long value
- if d > 127:
- d -= 256
- x = a | (b<<8) | (c<<16) | (d<<24)
- if x & 0x80000000 and x > 0:
- x = string.atoi(x - 0x100000000L)
- return x
-
- def r_long64(self):
- a = self.r_long()
- b = self.r_long()
- return a | (b<<32)
-
- def load_null(self):
- return NULL
- dispatch[TYPE_NULL] = load_null
-
- def load_none(self):
- return None
- dispatch[TYPE_NONE] = load_none
-
- def load_False(self):
- return False
- dispatch[TYPE_FALSE] = load_False
-
- def load_True(self):
- return True
- dispatch[TYPE_TRUE] = load_True
-
- def load_ellipsis(self):
- return EllipsisType
- dispatch[TYPE_ELLIPSIS] = load_ellipsis
-
- def load_int(self):
- return self.r_long()
- dispatch[TYPE_INT] = load_int
-
- def load_int64(self):
- return self.r_long64()
- dispatch[TYPE_INT64] = load_int64
-
- def load_long(self):
- size = self.r_long()
- sign = 1
- if size < 0:
- sign = -1
- size = -size
- x = 0L
- for i in range(size):
- d = self.r_short()
- x = x | (d<<(i*15L))
- return x * sign
- dispatch[TYPE_LONG] = load_long
-
- def load_float(self):
- n = ord(self.f.read(1))
- s = self.f.read(n)
- return string.atof(s)
- dispatch[TYPE_FLOAT] = load_float
-
- def load_complex(self):
- n = ord(self.f.read(1))
- s = self.f.read(n)
- real = float(s)
- n = ord(self.f.read(1))
- s = self.f.read(n)
- imag = float(s)
- return complex(real, imag)
- dispatch[TYPE_COMPLEX] = load_complex
-
- def load_string(self):
- n = self.r_long()
- return self.f.read(n)
- dispatch[TYPE_STRING] = load_string
-
- def load_tuple(self):
- return tuple(self.load_list())
- dispatch[TYPE_TUPLE] = load_tuple
-
- def load_list(self):
- n = self.r_long()
- list = []
- for i in range(n):
- list.append(self.load())
- return list
- dispatch[TYPE_LIST] = load_list
-
- def load_dict(self):
- d = {}
- while 1:
- key = self.load()
- if key is NULL:
- break
- value = self.load()
- d[key] = value
- return d
- dispatch[TYPE_DICT] = load_dict
-
- def load_code(self):
- argcount = self.r_short()
- nlocals = self.r_short()
- stacksize = self.r_short()
- flags = self.r_short()
- code = self.load()
- consts = self.load()
- names = self.load()
- varnames = self.load()
- filename = self.load()
- name = self.load()
- firstlineno = self.r_short()
- lnotab = self.load()
- if not new:
- raise RuntimeError, "can't unmarshal code objects; no 'new' module"
- return new.code(argcount, nlocals, stacksize, flags, code, consts,
- names, varnames, filename, name, firstlineno, lnotab)
- dispatch[TYPE_CODE] = load_code
-
-
-def dump(x, f):
- Marshaller(f).dump(x)
-
-def load(f):
- return Unmarshaller(f).load()
-
-def dumps(x):
- f = StringIO.StringIO()
- dump(x, f)
+def dumps(x, version=2):
+ f = cStringIO.StringIO()
+ dump(x, f, version)
return f.getvalue()
def loads(s):
- f = StringIO.StringIO(s)
+ f = cStringIO.StringIO(s)
return load(f)
Modified: trunk/jython/Lib/new.py
===================================================================
--- trunk/jython/Lib/new.py 2009-02-24 05:04:05 UTC (rev 6045)
+++ trunk/jython/Lib/new.py 2009-02-24 05:45:38 UTC (rev 6046)
@@ -20,3 +20,5 @@
# from types import CodeType as code
#except ImportError:
# pass
+
+from org.python.core import PyBytecode as code
Copied: trunk/jython/Lib/pycimport.py (from rev 6042, branches/pbcvm/Lib/pycimport.py)
===================================================================
--- trunk/jython/Lib/pycimport.py (rev 0)
+++ trunk/jython/Lib/pycimport.py 2009-02-24 05:45:38 UTC (rev 6046)
@@ -0,0 +1,84 @@
+import sys
+import os
+
+from org.python.core import imp as _imp, PyFrame as _Frame, Py as _Py
+from marshal import Unmarshaller
+
+__debugging__ = False
+
+def __readPycHeader(file):
+ def read():
+ return ord(file.read(1))
+ magic = read() | (read()<<8)
+ if not ( file.read(1) == '\r' and file.read(1) == '\n' ):
+ raise TypeError("not valid pyc-file")
+ mtime = read() | (read()<<8) | (read()<<16) | (read()<<24)
+ return magic, mtime
+
+def __makeModule(name, code, path):
+ module = _imp.addModule(name)
+ builtins = _Py.getSystemState().builtins
+ frame = _Frame(code, module.__dict__, module.__dict__, builtins)
+ module.__file__ = path
+ code.call(frame) # execute module code
+ return module
+
+class __Importer(object):
+ def __init__(self, path):
+ if __debugging__: print "Importer invoked"
+ self.__path = path
+ def find_module(self, fullname, path=None):
+ if __debugging__:
+ print "Importer.find_module(fullname=%s, path=%s)" % (
+ repr(fullname), repr(path))
+ path = fullname.split('.')
+ filename = path[-1]
+ path = path[:-1]
+ pycfile = os.path.join(self.__path, *(path + [filename + '.pyc']))
+ pyfile = os.path.join(self.__path, *(path + [filename + '.py']))
+ if os.path.exists(pycfile):
+ f = open(pycfile, 'rb')
+ try:
+ magic, mtime = __readPycHeader(f)
+ except:
+ return None # abort! not a valid pyc-file
+ f.close()
+ if os.path.exists(pyfile):
+ pytime = os.stat(pyfile).st_mtime
+ if pytime > mtime:
+ return None # abort! py-file was newer
+ return self
+ else:
+ return None # abort! pyc-file does not exist
+ def load_module(self, fullname):
+ path = fullname.split('.')
+ path[-1] += '.pyc'
+ filename = os.path.join(self.__path, *path)
+ f = open(filename, 'rb')
+ magic, mtime = __readPycHeader(f)
+ #code = Unmarshaller(f, magic=magic).load()
+ code = Unmarshaller(f).load()
+ if __debugging__: print "Successfully loaded:", fullname
+ return __makeModule( fullname, code, filename )
+
+class __MetaImporter(object):
+ def __init__(self):
+ self.__importers = {}
+ def find_module(self, fullname, path):
+ if __debugging__: print "MetaImporter.find_module(%s, %s)" % (
+ repr(fullname), repr(path))
+ for _path in sys.path:
+ if _path not in self.__importers:
+ try:
+ self.__importers[_path] = __Importer(_path)
+ except:
+ self.__importers[_path] = None
+ importer = self.__importers[_path]
+ if importer is not None:
+ loader = importer.find_module(fullname, path)
+ if loader is not None:
+ return loader
+ else:
+ return None
+
+sys.meta_path.insert(0, __MetaImporter())
Modified: trunk/jython/Lib/test/test_array.py
===================================================================
--- trunk/jython/Lib/test/test_array.py 2009-02-24 05:04:05 UTC (rev 6045)
+++ trunk/jython/Lib/test/test_array.py 2009-02-24 05:45:38 UTC (rev 6046)
@@ -213,7 +213,7 @@
self.assertRaises(TypeError, b.fromstring, 42)
b.fromstring(a.tostring())
self.assertEqual(a, b)
- if a.itemsize>1:
+ if a.itemsize>1 and self.typecode not in ('b', 'B'):
self.assertRaises(ValueError, b.fromstring, "x")
def test_repr(self):
Copied: trunk/jython/Lib/test/test_marshal.py (from rev 6042, branches/pbcvm/Lib/test/test_marshal.py)
===================================================================
--- trunk/jython/Lib/test/test_marshal.py (rev 0)
+++ trunk/jython/Lib/test/test_marshal.py 2009-02-24 05:45:38 UTC (rev 6046)
@@ -0,0 +1,264 @@
+#!/usr/bin/env python
+# -*- coding: iso-8859-1 -*-
+
+from __future__ import with_statement
+from test import test_support
+import marshal
+import sys
+import unittest
+import os
+
+# the original had incorrect semantics for non-refcounting GCs:
+# marshal.dump(expected, file(test_support.TESTFN, "wb"))
+# got = marshal.load(file(test_support.TESTFN, "rb"))
+
+def roundtrip(item):
+ with open(test_support.TESTFN, "wb") as test_file:
+ marshal.dump(item, test_file)
+ with open(test_support.TESTFN, "rb") as test_file:
+ got = marshal.load(file(test_support.TESTFN, "rb")) #, debug=True)
+ return got
+
+
+class IntTestCase(unittest.TestCase):
+ def test_ints(self):
+ # Test the full range of Python ints.
+ n = sys.maxint
+ while n:
+ for expected in (-n, n):
+ s = marshal.dumps(expected)
+ got = marshal.loads(s)
+ self.assertEqual(expected, got)
+ self.assertEqual(expected, roundtrip(expected))
+ n = n >> 1
+ os.unlink(test_support.TESTFN)
+
+ def test_int64(self):
+ # Simulate int marshaling on a 64-bit box. This is most interesting if
+ # we're running the test on a 32-bit box, of course.
+
+ def to_little_endian_string(value, nbytes):
+ bytes = []
+ for i in range(nbytes):
+ bytes.append(chr(value & 0xff))
+ value >>= 8
+ return ''.join(bytes)
+
+ maxint64 = (1L << 63) - 1
+ minint64 = -maxint64-1
+
+ for base in maxint64, minint64, -maxint64, -(minint64 >> 1):
+ while base:
+ s = 'I' + to_little_endian_string(base, 8)
+ got = marshal.loads(s)
+ self.assertEqual(base, got)
+ if base == -1: # a fixed-point for shifting right 1
+ base = 0
+ else:
+ base >>= 1
+
+ def test_bool(self):
+ for b in (True, False):
+ new = marshal.loads(marshal.dumps(b))
+ self.assertEqual(b, new)
+ self.assertEqual(type(b), type(new))
+ new = roundtrip(b)
+ self.assertEqual(b, new)
+ self.assertEqual(type(b), type(new))
+
+class FloatTestCase(unittest.TestCase):
+ def test_floats(self):
+ # Test a few floats
+ small = 1e-25
+ n = sys.maxint * 3.7e250
+ while n > small:
+ for expected in (-n, n):
+ f = float(expected)
+ s = marshal.dumps(f)
+ got = marshal.loads(s)
+ self.assertEqual(f, got)
+ got = roundtrip(f)
+ self.assertEqual(f, got)
+ n /= 123.4567
+
+ f = 0.0
+ s = marshal.dumps(f, 2)
+ got = marshal.loads(s)
+ self.assertEqual(f, got)
+ # and with version <= 1 (floats marshalled differently then)
+ s = marshal.dumps(f, 1)
+ got = marshal.loads(s)
+ self.assertEqual(f, got)
+
+ n = sys.maxint * 3.7e-250
+ while n < small:
+ for expected in (-n, n):
+ f = float(expected)
+
+ s = marshal.dumps(f)
+ got = marshal.loads(s)
+ self.assertEqual(f, got)
+
+ s = marshal.dumps(f, 1)
+ got = marshal.loads(s)
+ if test_support.is_jython:
+ self.assertAlmostEqual(f, got)
+ else:
+ self.assertEqual(f, got)
+
+ got = roundtrip(f)
+ self.assertEqual(f, got)
+
+ # XXX - not certain what this extra arg to dump is!
+ #marshal.dump(f, file(test_support.TESTFN, "wb"), 1)
+ #got = marshal.load(file(test_support.TESTFN, "rb"))
+ #self.assertEqual(f, got)
+ n *= 123.4567
+ os.unlink(test_support.TESTFN)
+
+class StringTestCase(unittest.TestCase):
+ def test_unicode(self):
+ for s in [u"", u"Andr\xE8 Previn", u"abc", u" "*10000]:
+ new = marshal.loads(marshal.dumps(s))
+ self.assertEqual(s, new)
+ self.assertEqual(type(s), type(new))
+ new = roundtrip(s)
+ self.assertEqual(s, new)
+ self.assertEqual(type(s), type(new))
+ os.unlink(test_support.TESTFN)
+
+ def test_string(self):
+ for s in ["", "Andr\xE8 Previn", "abc", " "*10000]:
+ new = marshal.loads(marshal.dumps(s))
+ self.assertEqual(s, new)
+ self.assertEqual(type(s), type(new))
+ new = roundtrip(s)
+ self.assertEqual(s, new)
+ self.assertEqual(type(s), type(new))
+ os.unlink(test_support.TESTFN)
+
+ if not test_support.is_jython:
+ def test_buffer(self):
+ for s in ["", "Andr\xE8 Previn", "abc", " "*10000]:
+ b = buffer(s)
+ new = marshal.loads(marshal.dumps(b))
+ self.assertEqual(s, new)
+ new = roundtrip(b)
+ self.assertEqual(s, new)
+ os.unlink(test_support.TESTFN)
+
+class ExceptionTestCase(unittest.TestCase):
+ def test_exceptions(self):
+ new = marshal.loads(marshal.dumps(StopIteration))
+ self.assertEqual(StopIteration, new)
+
+class CodeTestCase(unittest.TestCase):
+ if not test_support.is_jython: # XXX - need to use the PBC compilation backend, which doesn't exist yet
+ def test_code(self):
+ co = ExceptionTestCase.test_exceptions.func_code
+ new = marshal.loads(marshal.dumps(co))
+ self.assertEqual(co, new)
+
+class ContainerTestCase(unittest.TestCase):
+ d = {'astring': 'fo...@ba...z.spam',
+ 'afloat': 7283.43,
+ 'anint': 2**20,
+ 'ashortlong': 2L,
+ 'alist': ['.zyx.41'],
+ 'atuple': ('.zyx.41',)*10,
+ 'aboolean': False,
+ 'aunicode': u"Andr\xE8 Previn"
+ }
+ def test_dict(self):
+ new = marshal.loads(marshal.dumps(self.d))
+ self.assertEqual(self.d, new)
+ new = roundtrip(self.d)
+ self.assertEqual(self.d, new)
+ os.unlink(test_support.TESTFN)
+
+ def test_list(self):
+ lst = self.d.items()
+ new = marshal.loads(marshal.dumps(lst))
+ self.assertEqual(lst, new)
+ new = roundtrip(lst)
+ self.assertEqual(lst, new)
+ os.unlink(test_support.TESTFN)
+
+ def test_tuple(self):
+ t = tuple(self.d.keys())
+ new = marshal.loads(marshal.dumps(t))
+ self.assertEqual(t, new)
+ new = roundtrip(t)
+ self.assertEqual(t, new)
+ os.unlink(test_support.TESTFN)
+
+ def test_sets(self):
+ for constructor in (set, frozenset):
+ t = constructor(self.d.keys())
+ new = marshal.loads(marshal.dumps(t))
+ self.assertEqual(t, new)
+ self.assert_(isinstance(new, constructor))
+ self.assertNotEqual(id(t), id(new))
+ new = roundtrip(t)
+ self.assertEqual(t, new)
+ os.unlink(test_support.TESTFN)
+
+class BugsTestCase(unittest.TestCase):
+ def test_bug_5888452(self):
+ # Simple-minded check for SF 588452: Debug build crashes
+ marshal.dumps([128] * 1000)
+
+ def test_patch_873224(self):
+ self.assertRaises(Exception, marshal.loads, '0')
+ self.assertRaises(Exception, marshal.loads, 'f')
+ self.assertRaises(Exception, marshal.loads, marshal.dumps(5L)[:-1])
+
+ def test_version_argument(self):
+ # Python 2.4.0 crashes for any call to marshal.dumps(x, y)
+ self.assertEquals(marshal.loads(marshal.dumps(5, 0)), 5)
+ self.assertEquals(marshal.loads(marshal.dumps(5, 1)), 5)
+
+ def test_fuzz(self):
+ # simple test that it's at least not *totally* trivial to
+ # crash from bad marshal data
+ for c in [chr(i) for i in range(256)]:
+ try:
+ marshal.loads(c)
+ except Exception:
+ pass
+
+ def test_loads_recursion(self):
+ s = 'c' + ('X' * 4*4) + '{' * 2**20
+ self.assertRaises(ValueError, marshal.loads, s)
+
+ def test_recursion_limit(self):
+ # Create a deeply nested structure.
+ head = last = []
+ # The max stack depth should match the value in Python/marshal.c.
+ MAX_MARSHAL_STACK_DEPTH = 2000
+ for i in range(MAX_MARSHAL_STACK_DEPTH - 2):
+ last.append([0])
+ last = last[-1]
+
+ # Verify we don't blow out the stack with dumps/load.
+ data = marshal.dumps(head)
+ new_head = marshal.loads(data)
+ # Don't use == to compare objects, it can exceed the recursion limit.
+ self.assertEqual(len(new_head), len(head))
+ self.assertEqual(len(new_head[0]), len(head[0]))
+ self.assertEqual(len(new_head[-1]), len(head[-1]))
+
+ last.append([0])
+ self.assertRaises(ValueError, marshal.dumps, head)
+
+def test_main():
+ test_support.run_unittest(IntTestCase,
+ FloatTestCase,
+ StringTestCase,
+ CodeTestCase,
+ ContainerTestCase,
+ ExceptionTestCase,
+ BugsTestCase)
+
+if __name__ == "__main__":
+ test_main()
Deleted: trunk/jython/Tools/pbcvm/extract.py
===================================================================
--- branches/pbcvm/Tools/pbcvm/extract.py 2009-02-24 04:25:13 UTC (rev 6042)
+++ trunk/jython/Tools/pbcvm/extract.py 2009-02-24 05:45:38 UTC (rev 6046)
@@ -1,141 +0,0 @@
-"""Given a module, generates a new module where all functions that are
- not builtin (including within classes) have their func_code pointed
- to a PyBytecode constructor. This enables CPython to generate for
- Jython the desired PBC.
-"""
-
-from __future__ import with_statement
-import inspect
-import networkx # add dependency to a setup script? probably overkill
-import sys
-from collections import defaultdict
-
-attrs = ["co_" + x for x in """
- argcount nlocals stacksize flags
- code consts names varnames
- filename name firstlineno lnotab
- freevars cellvars
-""".split()] # add
-
-ROOT = object()
-
-class Extract(object):
-
- def __init__(self, mod, writer):
- self.mod = mod
- self.writer = writer
- self.codeobjs = {}
- self.codeobjs_names = {}
- self.counters = defaultdict(int)
- self.depend = networkx.DiGraph()
-
- def extract(self):
- mod = self.mod
- writer = self.writer
- functionobjs = self.candidate_functions()
- for name, f in functionobjs:
- self.extract_code_obj(f)
-
- print >> writer, "from %s import *" % mod.__name__
- print >> writer, "from org.python.core import PyBytecode"
- print >> writer
- print >> writer, "_codeobjs = {}"
- print >> writer
-
- objs = networkx.topological_sort(self.depend)
- for obj in objs:
- if not inspect.iscode(obj):
- continue
- name = self.codeobjs_names[obj]
- print >> writer, "_codeobjs[%r] = %s" % (name, self.codeobjs[obj])
- print >> writer
- for name, f in functionobjs:
- # this may be a Jython diff, need to determine further; need to check if im_func or not on the object
- print >> writer, "try: %s.func_code = _codeobjs[%r]" % (name, self.codeobjs_names[f.func_code])
- print >> writer, "except (AttributeError, ValueError): pass" # ignore setting cells, im_func, etc... %s.im_func.func_code = _codeobjs[%r]" % (name, self.codeobjs_names[f.func_code])
-
- print >> writer
- print >> writer, 'if __name__ == "__main__":'
- print >> writer, ' test_main()'
-
- def candidate_functions(self):
- """functions and methods we will retrieve code objects"""
-
- mod = self.mod
- functions = inspect.getmembers(mod, inspect.isfunction)
- functions = [(name, f) for (name, f) in functions if not inspect.isbuiltin(f) and name != "test_main"]
-
- classes = inspect.getmembers(mod, inspect.isclass)
- for classname, cls in classes:
- for methodname, method in inspect.getmembers(cls, inspect.ismethod):
- if inspect.getmodule(method) == mod:
- functions.append(("%s.%s" % (classname, methodname), method))
- return functions
-
- def extract_code_obj(self, f_or_code):
- if inspect.iscode(f_or_code):
- code = f_or_code
- else:
- code = f_or_code.func_code
- self.extract_def(code)
-
- def extract_def(self, code):
- if code in self.codeobjs_names:
- #print >> sys.stderr, "Already seen", code
- return "_codeobjs[%r]" % (self.codeobjs_names[code],)
-
- co_name = code.co_name
- #print >> sys.stderr, "Processing", code
- name = co_name + "." + str(self.counters[co_name])
- self.counters[co_name] += 1
- self.codeobjs_names[code] = name
- values = []
- self.depend.add_edge(code, ROOT)
- for attr in attrs:
- # treat co_consts specially - maybe also need to use pickling in case repr is not suitable
- if attr == 'co_consts':
- co_consts = []
- for const in getattr(code, attr):
- if inspect.iscode(const):
- #print >> sys.stderr, "Extracting code const " + str(const)
- co_consts.append(self.extract_def(const))
- self.depend.add_edge(const, code)
- else:
- co_consts.append(repr(const))
- values.append((attr, "["+', '.join(co_consts)+"]"))
- else:
- values.append((attr, repr(getattr(code, attr))))
- self.codeobjs[code] = "PyBytecode(\n" + '\n'.join([' '* 4 + v + ', # ' + attr for (attr, v) in values])+"\n )"
- return "_codeobjs[%r]" % (name,)
-
-
-# for now, just use the cwd
-def import_modules(path):
- import glob
- import os.path
-
- sys.path.insert(0, path)
- for name in sorted(glob.iglob("test/test_*.py")):
- modname = os.path.splitext(os.path.basename(name))[0]
- qualified_name = "test." + modname
- print "Trying", qualified_name
- try:
- topmod = __import__(qualified_name)
- except Exception, e:
- print "Could not import", qualified_name, ":", repr(e)
- continue
- mod = getattr(topmod, modname)
- try:
- mod.test_main
- except AttributeError, e:
- print "No test_main in", qualified_name
- continue
-
- output_name = "test/" + modname + "_pbc.py"
- with open(output_name, "w") as f:
- print "Extracting", name, "to", output_name
- Extract(mod, f).extract()
-
-if __name__ == '__main__':
- path = modname = sys.argv[1]
- import_modules(path)
Copied: trunk/jython/Tools/pbcvm/extract.py (from rev 6042, branches/pbcvm/Tools/pbcvm/extract.py)
===================================================================
--- trunk/jython/Tools/pbcvm/extract.py (rev 0)
+++ trunk/jython/Tools/pbcvm/extract.py 2009-02-24 05:45:38 UTC (rev 6046)
@@ -0,0 +1,141 @@
+"""Given a module, generates a new module where all functions that are
+ not builtin (including within classes) have their func_code pointed
+ to a PyBytecode constructor. This enables CPython to generate for
+ Jython the desired PBC.
+"""
+
+from __future__ import with_statement
+import inspect
+import networkx # add dependency to a setup script? probably overkill
+import sys
+from collections import defaultdict
+
+attrs = ["co_" + x for x in """
+ argcount nlocals stacksize flags
+ code consts names varnames
+ filename name firstlineno lnotab
+ freevars cellvars
+""".split()] # add
+
+ROOT = object()
+
+class Extract(object):
+
+ def __init__(self, mod, writer):
+ self.mod = mod
+ self.writer = writer
+ self.codeobjs = {}
+ self.codeobjs_names = {}
+ self.counters = defaultdict(int)
+ self.depend = networkx.DiGraph()
+
+ def extract(self):
+ mod = self.mod
+ writer = self.writer
+ functionobjs = self.candidate_functions()
+ for name, f in functionobjs:
+ self.extract_code_obj(f)
+
+ print >> writer, "from %s import *" % mod.__name__
+ print >> writer, "from org.python.core import PyBytecode"
+ print >> writer
+ print >> writer, "_codeobjs = {}"
+ print >> writer
+
+ objs = networkx.topological_sort(self.depend)
+ for obj in objs:
+ if not inspect.iscode(obj):
+ continue
+ name = self.codeobjs_names[obj]
+ print >> writer, "_codeobjs[%r] = %s" % (name, self.codeobjs[obj])
+ print >> writer
+ for name, f in functionobjs:
+ # this may be a Jython diff, need to determine further; need to check if im_func or not on the object
+ print >> writer, "try: %s.func_code = _codeobjs[%r]" % (name, self.codeobjs_names[f.func_code])
+ print >> writer, "except (AttributeError, ValueError): pass" # ignore setting cells, im_func, etc... %s.im_func.func_code = _codeobjs[%r]" % (name, self.codeobjs_names[f.func_code])
+
+ print >> writer
+ print >> writer, 'if __name__ == "__main__":'
+ print >> writer, ' test_main()'
+
+ def candidate_functions(self):
+ """functions and methods we will retrieve code objects"""
+
+ mod = self.mod
+ functions = inspect.getmembers(mod, inspect.isfunction)
+ functions = [(name, f) for (name, f) in functions if not inspect.isbuiltin(f) and name != "test_main"]
+
+ classes = inspect.getmembers(mod, inspect.isclass)
+ for classname, cls in classes:
+ for methodname, method in inspect.getmembers(cls, inspect.ismethod):
+ if inspect.getmodule(method) == mod:
+ functions.append(("%s.%s" % (classname, methodname), method))
+ return functions
+
+ def extract_code_obj(self, f_or_code):
+ if inspect.iscode(f_or_code):
+ code = f_or_code
+ else:
+ code = f_or_code.func_code
+ self.extract_def(code)
+
+ def extract_def(self, code):
+ if code in self.codeobjs_names:
+ #print >> sys.stderr, "Already seen", code
+ return "_codeobjs[%r]" % (self.codeobjs_names[code],)
+
+ co_name = code.co_name
+ #print >> sys.stderr, "Processing", code
+ name = co_name + "." + str(self.counters[co_name])
+ self.counters[co_name] += 1
+ self.codeobjs_names[code] = name
+ values = []
+ self.depend.add_edge(code, ROOT)
+ for attr in attrs:
+ # treat co_consts specially - maybe also need to use pickling in case repr is not suitable
+ if attr == 'co_consts':
+ co_consts = []
+ for const in getattr(code, attr):
+ if inspect.iscode(const):
+ #print >> sys.stderr, "Extracting code const " + str(const)
+ co_consts.append(self.extract_def(const))
+ self.depend.add_edge(const, code)
+ else:
+ co_consts.append(repr(const))
+ values.append((attr, "["+', '.join(co_consts)+"]"))
+ else:
+ values.append((attr, repr(getattr(code, attr))))
+ self.codeobjs[code] = "PyBytecode(\n" + '\n'.join([' '* 4 + v + ', # ' + attr for (attr, v) in values])+"\n )"
+ return "_codeobjs[%r]" % (name,)
+
+
+# for now, just use the cwd
+def import_modules(path):
+ import glob
+ import os.path
+
+ sys.path.insert(0, path)
+ for name in sorted(glob.iglob("test/test_*.py")):
+ modname = os.path.splitext(os.path.basename(name))[0]
+ qualified_name = "test." + modname
+ print "Trying", qualified_name
+ try:
+ topmod = __import__(qualified_name)
+ except Exception, e:
+ print "Could not import", qualified_name, ":", repr(e)
+ continue
+ mod = getattr(topmod, modname)
+ try:
+ mod.test_main
+ except AttributeError, e:
+ print "No test_main in", qualified_name
+ continue
+
+ output_name = "test/" + modname + "_pbc.py"
+ with open(output_name, "w") as f:
+ print "Extracting", name, "to", output_name
+ Extract(mod, f).extract()
+
+if __name__ == '__main__':
+ path = modname = sys.argv[1]
+ import_modules(path)
Deleted: trunk/jython/src/org/python/compiler/pbc/Bytecode.java
===================================================================
--- branches/pbcvm/src/org/python/compiler/pbc/Bytecode.java 2009-02-24 04:25:13 UTC (rev 6042)
+++ trunk/jython/src/org/python/compiler/pbc/Bytecode.java 2009-02-24 05:45:38 UTC (rev 6046)
@@ -1,555 +0,0 @@
-package org.python.compiler.pbc;
-
-// Copyright (c) 2009 Jython Developers
-//
-// ported from peak.util.assembler (BytecodeAssembler):
-// Copyright (C) 1996-2004 by Phillip J. Eby and Tyler C. Sarna.
-// All rights reserved. This software may be used under the same terms
-// as Zope or Python. (http://cvs.eby-sarna.com/*checkout*/PEAK/README.txt)
-//
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.LinkedHashMap;
-import java.util.LinkedHashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-import org.python.core.Py;
-import org.python.core.Opcode;
-import org.python.core.PyBaseCode;
-import org.python.core.PyBytecode;
-import org.python.core.PyObject;
-
-// must be thread confined, as might be expected
-class Bytecode {
-
- private int co_argcount = 0;
- private int co_stacksize = 0;
- private int co_flags = PyBaseCode.CO_OPTIMIZED | PyBaseCode.CO_NEWLOCALS; // typical usage
-// co_filename = '<generated code>'
-// co_name = '<lambda>'
-// co_firstlineno = 0
-// co_freevars = ()
-// co_cellvars = ()
-// _last_lineofs = 0
-// _ss = 0
-// _tmp_level = 0
-//
-
- public Bytecode(int co_flags) {
- this.co_flags = co_flags;
- co_const.put(Py.None, 0);
- }
-// def __init__(self):
-// self.co_code = array('B')
-// self.co_names = []
-// self.co_varnames = []
-// self.blocks = []
- private final List<Integer> stack_history = new ArrayList<Integer>();
- private final List<Integer> co_code = new ArrayList<Integer>();
- private final List<Integer> co_lnotab = new ArrayList<Integer>();
- private final Map<PyObject, Integer> co_const = new HashMap<PyObject, Integer>();
-
- private void emit(int opcode) {
- assert (opcode >= 0 && opcode <= 0xFF); // at this point we should verify we are only emitting unsigned bytes
- co_code.add(opcode);
- }
-
- private void emit(int opcode, int oparg) {
- if (oparg > 0xFFFF) {
- emit(Opcode.EXTENDED_ARG);
- emit((oparg >> 16) & 0xFF);
- emit((oparg >> 24) & 0xFF);
- }
- emit(opcode);
- emit(oparg & 0xFF);
- emit((oparg >> 8) & 0xFF);
- }
-
- private void LOAD_CONST(PyObject constant) {
- int arg;
- if (co_const.containsKey(constant)) {
- arg = co_const.get(constant);
- } else {
- arg = co_const.size() + 1;
- co_const.put(constant, arg);
- }
- stackchange(0, 1);
- emit(Opcode.LOAD_CONST, arg);
- }
-
- private void RETURN_VALUE() {
- stackchange(1, 0);
- emit(Opcode.RETURN_VALUE);
- stack_unknown();
- }
-
- public void code() {
-// return new PyBytecode();
- }
-
-
-//
-
-//
-// def locals_written(self):
-// vn = self.co_varnames
-// hl = dict.fromkeys([STORE_FAST, DELETE_FAST])
-// return dict.fromkeys([vn[arg] for ofs, op, arg in self if op in hl])
-//
-//
-//
-//
- public void set_lineno(int lno) {
-// if not self.co_firstlineno:
-// self.co_firstlineno = self._last_line = lno
-// return
-//
-// append = self.co_lnotab.append
-// incr_line = lno - self._last_line
-// incr_addr = len(self.co_code) - self._last_lineofs
-// if not incr_line:
-// return
-//
-// assert incr_addr>=0 and incr_line>=0
-//
-// while incr_addr>255:
-// append(255)
-// append(0)
-// incr_addr -= 255
-//
-// while incr_line>255:
-// append(incr_addr)
-// append(255)
-// incr_line -= 255
-// incr_addr = 0
-//
-// if incr_addr or incr_line:
-// append(incr_addr)
-// append(incr_line)
-//
-// self._last_line = lno
-// self._last_lineofs = len(self.co_code)
- }
- //
-
- public void YIELD_VALUE() {
- stackchange(1, 1);
- co_flags |= PyBaseCode.CO_GENERATOR;
- emit(Opcode.YIELD_VALUE);
- }
-
- // defaults? def CALL_FUNCTION(self, argc=0, kwargc=0, op=CALL_FUNCTION, extra=0):
- public void CALL_FUNCTION(int argc, int kwargc, int op, int extra) {
- stackchange(1 + argc + 2 * kwargc + extra, 1);
- emit(op);
- emit(argc);
- emit(kwargc);
- }
-//
-// def CALL_FUNCTION_VAR(self, argc=0, kwargc=0):
-// self.CALL_FUNCTION(argc,kwargc,CALL_FUNCTION_VAR, 1) # 1 for *args
-//
-// def CALL_FUNCTION_KW(self, argc=0, kwargc=0):
-// self.CALL_FUNCTION(argc,kwargc,CALL_FUNCTION_KW, 1) # 1 for **kw
-//
-// def CALL_FUNCTION_VAR_KW(self, argc=0, kwargc=0):
-// self.CALL_FUNCTION(argc,kwargc,CALL_FUNCTION_VAR_KW, 2) # 2 *args,**kw
-//
-
- public void BUILD_TUPLE(int count) {
- stackchange(count, 1);
- emit(Opcode.BUILD_TUPLE, count);
- }
-
- public void BUILD_LIST(int count) {
- stackchange(count, 1);
- emit(Opcode.BUILD_LIST, count);
- }
-
- public void UNPACK_SEQUENCE(int count) {
- stackchange(1, count);
- emit(Opcode.UNPACK_SEQUENCE, count);
- }
-
- public void BUILD_SLICE(int count) {
- if (count != 2 && count != 3) {
- throw Py.AssertionError("Invalid number of arguments for BUILD_SLICE");
- }
- stackchange(count, 1);
- emit(Opcode.BUILD_SLICE, count);
- }
-
- public void DUP_TOPX(int count) {
- stackchange(count, count * 2);
- emit(Opcode.DUP_TOPX, count);
- }
-
- public void RAISE_VARARGS(int argc) {
- if (0 <= argc && argc <= 3) {
- throw Py.AssertionError("Invalid number of arguments for RAISE_VARARGS");
- }
- stackchange(argc, 0);
- emit(Opcode.RAISE_VARARGS, argc);
- }
-
- public void MAKE_FUNCTION(int ndefaults) {
- stackchange(1 + ndefaults, 1);
- emit(Opcode.MAKE_FUNCTION, ndefaults);
- }
-
- public void MAKE_CLOSURE(int ndefaults, int freevars) {
- freevars = 1;
- stackchange(1 + freevars + ndefaults, 1);
- emit(Opcode.MAKE_CLOSURE, ndefaults);
- }
-
- public int here() {
- return co_code.size();
- }
- private int _ss = -1;
-
- public void set_stack_size(int size) {
- if (size < 0) {
- throw Py.AssertionError("Stack underflow");
- }
- if (size > co_stacksize) {
- co_stacksize = size;
- }
- int bytes = co_code.size() - stack_history.size() + 1;
- if (bytes > 0) {
- for (int i = 0; i < bytes; i++) {
- stack_history.add(_ss);
- }
- }
- _ss = size;
- }
-
- public int get_stack_size() {
- return _ss;
- }
-
- public void stackchange(int inputs, int outputs) {
- if (_ss == -1) {
- throw Py.AssertionError("Unknown stack size at this location");
- }
- set_stack_size(get_stack_size() - inputs); // check underflow
- set_stack_size(get_stack_size() + outputs); // update maximum height
-
- }
-
- public void stack_unknown() {
- _ss = -1;
- }
-
- public void branch_stack(int location, int expected) {
- if (location >= stack_history.size()) {
- if (location > co_code.size()) {
- throw Py.AssertionError(String.format(
- "Forward-looking stack prediction! %d, %d", location, co_code.size()));
- }
- int actual = get_stack_size();
- if (actual == -1) {
- actual = expected;
- set_stack_size(actual);
- stack_history.set(location, actual);
- } else {
- actual = stack_history.get(location);
- if (actual == -1) {
- actual = expected;
- stack_history.set(location, actual);
- }
- if (actual != expected) {
- throw Py.AssertionError(String.format(
- "Stack level mismatch: actual=%d expected=%d", actual, expected));
- }
- }
- }
- }
-
-// def jump(self, op, arg=None):
-// def jump_target(offset):
-// target = offset
-// if op not in hasjabs:
-// target = target - (posn+3)
-// assert target>=0, "Relative jumps can't go backwards"
-// if target>0xFFFF:
-// target = offset - (posn+6)
-// return target
-//
-// def backpatch(offset):
-// target = jump_target(offset)
-// if target>0xFFFF:
-// raise AssertionError("Forward jump span must be <64K bytes")
-// self.patch_arg(posn, 0, target)
-// self.branch_stack(offset, old_level)
-//
-// if op==FOR_ITER:
-// old_level = self.stack_size = self.stack_size - 1
-// self.stack_size += 2
-// else:
-// old_level = self.stack_size
-// posn = self.here()
-//
-// if arg is not None:
-// self.emit_arg(op, jump_target(arg))
-// self.branch_stack(arg, old_level)
-// lbl = None
-// else:
-// self.emit_arg(op, 0)
-// def lbl(code=None):
-// backpatch(self.here())
-// if op in (JUMP_FORWARD, JUMP_ABSOLUTE, CONTINUE_LOOP):
-// self.stack_unknown()
-// return lbl
-//
-// def COMPARE_OP(self, op):
-// self.stackchange((2,1))
-// self.emit_arg(COMPARE_OP, compares[op])
-//
-//
-// def setup_block(self, op):
-// jmp = self.jump(op)
-// self.blocks.append((op,self.stack_size,jmp))
-// return jmp
-//
-// def SETUP_EXCEPT(self):
-// ss = self.stack_size
-// self.stack_size = ss+3 # simulate the level at "except:" time
-// self.setup_block(SETUP_EXCEPT)
-// self.stack_size = ss # restore the current level
-//
-// def SETUP_FINALLY(self):
-// ss = self.stack_size
-// self.stack_size = ss+3 # allow for exceptions
-// self.stack_size = ss+1 # simulate the level after the None is pushed
-// self.setup_block(SETUP_FINALLY)
-// self.stack_size = ss # restore original level
-//
-// def SETUP_LOOP(self):
-// self.setup_block(SETUP_LOOP)
-//
-// def POP_BLOCK(self):
-// if not self.blocks:
-// raise AssertionError("Not currently in a block")
-//
-// why, level, fwd = self.blocks.pop()
-// self.emit(POP_BLOCK)
-//
-// if why!=SETUP_LOOP:
-// if why==SETUP_FINALLY:
-// self.LOAD_CONST(None)
-// fwd()
-// else:
-// self.stack_size = level-3 # stack level resets here
-// else_ = self.JUMP_FORWARD()
-// fwd()
-// return else_
-// else:
-// return fwd
-//
-//
-// def assert_loop(self):
-// for why,level,fwd in self.blocks:
-// if why==SETUP_LOOP:
-// return
-// raise AssertionError("Not inside a loop")
-//
-// def BREAK_LOOP(self):
-// self.assert_loop(); self.emit(BREAK_LOOP)
-// self.stack_unknown()
-//
-// def CONTINUE_LOOP(self, label):
-// self.assert_loop()
-// if self.blocks[-1][0]==SETUP_LOOP:
-// op = JUMP_ABSOLUTE # more efficient if not in a nested block
-// else:
-// op = CONTINUE_LOOP
-// return self.jump(op, label)
-//
-// def __call__(self, *args):
-// last = None
-// for ob in args:
-// if callable(ob):
-// last = ob(self)
-// else:
-// try:
-// f = generate_types[type(ob)]
-// except KeyError:
-// raise TypeError("Can't generate", ob)
-// else:
-// last = f(self, ob)
-// return last
-//
-// def return_(self, ob=None):
-// return self(ob, Code.RETURN_VALUE)
-//
-// decorate(classmethod)
-// def from_function(cls, function, copy_lineno=False):
-// code = cls.from_code(function.func_code, copy_lineno)
-// return code
-//
-//
-// decorate(classmethod)
-// def from_code(cls, code, copy_lineno=False):
-// import inspect
-// self = cls.from_spec(code.co_name, *inspect.getargs(code))
-// if copy_lineno:
-// self.set_lineno(code.co_firstlineno)
-// self.co_filename = code.co_filename
-// self.co_freevars = code.co_freevars # XXX untested!
-// return self
-//
-// decorate(classmethod)
-// def from_spec(cls, name='<lambda>', args=(), var=None, kw=None):
-// self = cls()
-// self.co_name = name
-// self.co_argcount = len(args)
-// self.co_varnames.extend(args)
-// if var:
-// self.co_varnames.append(var)
-// self.co_flags |= CO_VARARGS
-// if kw:
-// self.co_varnames.append(kw)
-// self.co_flags |= CO_VARKEYWORDS
-//
-// def tuple_arg(args):
-// self.UNPACK_SEQUENCE(len(args))
-// for arg in args:
-// if not isinstance(arg, basestring):
-// tuple_arg(arg)
-// else:
-// self.STORE_FAST(arg)
-//
-// for narg, arg in enumerate(args):
-// if not isinstance(arg, basestring):
-// dummy_name = '.'+str(narg)
-// self.co_varnames[narg] = dummy_name
-// self.LOAD_FAST(dummy_name)
-// tuple_arg(arg)
-//
-// return self
-//
-//
-// def patch_arg(self, offset, oldarg, newarg):
-// code = self.co_code
-// if (oldarg>0xFFFF) != (newarg>0xFFFF):
-// raise AssertionError("Can't change argument size", oldarg, newarg)
-// code[offset+1] = newarg & 255
-// code[offset+2] = (newarg>>8) & 255
-// if newarg>0xFFFF:
-// newarg >>=16
-// code[offset-2] = newarg & 255
-// code[offset-1] = (newarg>>8) & 255
-//
-// def nested(self, name='<lambda>', args=(), var=None, kw=None, cls=None):
-// if cls is None:
-// cls = Code
-// code = cls.from_spec(name, args, var, kw)
-// code.co_filename=self.co_filename
-// return code
-//
-// def __iter__(self):
-// i = 0
-// extended_arg = 0
-// code = self.co_code
-// n = len(code)
-// while i < n:
-// op = code[i]
-// if op >= HAVE_ARGUMENT:
-// oparg = code[i+1] + code[i+2]*256 + extended_arg
-// extended_arg = 0
-// if op == EXTENDED_ARG:
-// extended_arg = oparg*65536
-// i+=3
-// continue
-// yield i, op, oparg
-// i += 3
-// else:
-// yield i, op, None
-// i += 1
-//
-//
-//
-//
-// def makefree(self, names):
-// nowfree = dict.fromkeys(self.co_freevars)
-// newfree = [n for n in names if n not in nowfree]
-// if newfree:
-// self.co_freevars += tuple(newfree)
-// self._locals_to_cells()
-//
-// def makecells(self, names):
-// nowcells = dict.fromkeys(self.co_cellvars+self.co_freevars)
-// newcells = [n for n in names if n not in nowcells]
-// if new...
[truncated message content] |
