Thanks a lot, Philip.  That resolved the problem.  Now I'm calling types.FunctionType with the following arguments instead of calling the PyFunction constructor.

      clone[key] = types.FunctionType(
          clone[key].func_code,
          clone,  # globals
          clone[key].func_name,
          clone[key].func_defaults,
          clone[key].func_closure)

Unfortunately, now I've run into another problem where my application is failing with the following error being logged when a new PythonInterpreter tries to run some Python code:


SystemError: Automatic proxy initialization should only occur on proxy classes

I'll try to do some more debugging to find out what is causing this error.

--------------------------------------------------------------
Sharon Lucas
IBM Austin,   lucass@us.ibm.com
(512) 286-7313 or Tieline 363-7313



Philip Jenvey <pjenvey@underboss.org>

01/18/2010 04:12 PM

To
Sharon Lucas/Austin/IBM@IBMUS
cc
jython-users@lists.sourceforge.net
Subject
Re: [Jython-users] "TypeError: function() argument 1 must be code,        not stringmap" when calling org.python.core.PyFunction()






On Jan 18, 2010, at 1:52 PM, Sharon Lucas wrote:

>
> Hi,
>
> We have a Java application that uses Jython and works fine with Jython 2.1.  I'm now porting it to use Jython 2.5.1.  However, I'm getting the following error using Jython 2.5.1
>
>    "TypeError: function() argument 1 must be code, not stringmap"
>
> in our Jython function that does our version of a "deep copy" of a PyFunction definition where we create an identical PyFunction, except for the reference to the global variable pool (which is redirected to look at a new global variable pool).   Here is a simple Jython program I wrote that recreates this problem.  It defines a function named cloneGlobals and a main function that calls this function passing it globals() as the argument.  When run via Jython 2.1, it works fine.  However, when run via Jython 2.5.1, it fails when calling org.python.core.PyFunction (the line in blue).
>
> def cloneGlobals(theGlobals):
>   import copy
>   import types
>   import org.python.core.PyFunction
>      
>   clone = copy.copy(theGlobals)
>   skipTypes = [types.ModuleType, types.FunctionType,
>                types.ClassType,
>                type(clone) # StringMapType
>               ]
>
>   for key in clone.keys():
>     if (type(clone[key]) not in skipTypes):
>       try:
>         clone[key] = copy.deepcopy(clone[key])
>       except:
>         pass  # ignore types that cannot be deep copied
>     elif ((type(clone[key]) is types.FunctionType) and
>           (clone[key].func_globals is theGlobals)):
>
>       # Checks to see if the variable is a function definition that was
>       # defined at the "global" scope, i.e., not within some other
>       # Python module.  If so, then it replaces the function definition
>       # with one that is identical except for the reference to the global
>       # variable pool, which is redirected to look at the new global
>       # variable pool being created.
>
>       print '\nFunction name: %s' % (key)
>       print '  globals : %s' % (type(clone))
>       print '  defaults: %s' % (type(clone[key].func_defaults))
>       print '  code    : %s' % (type(clone[key].func_code))
>       print '  doc     : %s' % (type(clone[key].func_doc))
>       print '  closure : %s' % (type(clone[key].func_closure))
>       clone[key] = org.python.core.PyFunction(
>           clone,  # globals
>           clone[key].func_defaults,
>           clone[key].func_code,
>           clone[key].func_doc,
>           clone[key].func_closure)
>
>   return clone
>
> def main():
>     print "\nglobals() = %s" % globals()
>     myClonedGlobals = cloneGlobals(globals())
>     print "\nmyClonedGlobals = %s" % (myClonedGlobals)
>
> if __name__ == '__main__':
>     main()
>
> Here's the output when running this code using Jython 2.1 which works:
>
> C:\Jython-2.1>jython C:\test\testPyFunction.py
>
> globals() = {'__doc__': None, 'main': <function main at 2823864>, '__name__': '__main__', 'cloneGlobals': <function cloneGlobals at 14729340>}
>
> Function name: main
>   globals : org.python.core.PyStringMap
>   defaults: org.python.core.PyNone
>   code    : org.python.core.PyTableCode
>   doc     : org.python.core.PyNone
>   closure : org.python.core.PyNone
>
> Function name: cloneGlobals
>   globals : org.python.core.PyStringMap
>   defaults: org.python.core.PyNone
>   code    : org.python.core.PyTableCode
>   doc     : org.python.core.PyNone
>   closure : org.python.core.PyNone
>
> myClonedGlobals = {'__name__': '__main__', 'cloneGlobals': <function cloneGlobals at 9291303>, '__doc__': None, 'main': <function main at 30169840>}
>
> Here's the output when running this code using Jython 2.5.1 which fails:
>
> C:\jython2.5.1>jython C:\test\testPyFunction.py
>
> globals() = {'__file__': 'C:\\test\\testPyFunction.py', '__doc__': None, 'main': <function main at 0x1>, '__name__': '__main__', 'cloneGlobals': <function cloneGlobals at 0x2>}
>
> Function name: main
>   globals : <type 'stringmap'>
>   defaults: <type 'NoneType'>
>   code    : <type 'tablecode'>
>   doc     : <type 'NoneType'>
>   closure : <type 'NoneType'>
> Traceback (most recent call last):
>   File "C:\dev\src\stax\testPyFunction.py", line 49, in <module>
>     main()
>   File "C:\dev\src\stax\testPyFunction.py", line 45, in main
>     myClonedGlobals = cloneGlobals(globals())
>   File "C:\dev\src\stax\testPyFunction.py", line 34, in cloneGlobals
>     clone[key] = org.python.core.PyFunction(
> TypeError: function() argument 1 must be code, not stringmap
>
> C:\jython2.5.1>
>
> When we call org.python.core.PyFunction(), we pass <type 'stringmap'> for globals, <type 'NoneType'> for defaults, <type 'tablecode'> for code, <type 'NoneType'> for doc, and <type 'NoneType'> for closure, in that order as defined by the PyFunction constructor, as shown in the output above.  The error message says the first argument must be of type code (aka. PyCode), not stringmap.  But it appears to me when looking at the source code for PyFunction.java in Jython 2.5.1  that the first argument passed to the PyFunction constructor should be for globals (for which a stringmap type argument should be fine) and the third argument should be for code (for which a tablecode type argument should be fine since PyTableCode extends PyCode).  But it is checking if the first argument is of type PyCode which appears to be causing the error.
>
> The source code for PyFunction.java in Jython 2.5.1 defines the PyFunction constructor as follows (as it did in Jython 2.1).  This is the constructor that we're calling.
>
>     public PyFunction(PyObject globals, PyObject[] defaults, PyCode code, PyObject doc,
>                       PyObject[] closure_cells) {
>
> The source code for PyFunction.java in Jython 2.5.1 also contains the following method that throws the error that we are getting.  
>
>     @ExposedNew
>     static final PyObject function___new__(PyNewWrapper new_, boolean init, PyType subtype,
>                                            PyObject[] args, String[] keywords) {
>         ArgParser ap = new ArgParser("function", args, keywords,
>                                      new String[] {"code", "globals", "name", "argdefs",
>                                                    "closure"}, 0);
>         PyObject code = ap.getPyObject(0);
>         PyObject globals = ap.getPyObject(1);
>         PyObject name = ap.getPyObject(2, Py.None);
>         PyObject defaults = ap.getPyObject(3, Py.None);
>         PyObject closure = ap.getPyObject(4, Py.None);
>
>         if (!(code instanceof PyBaseCode)) {
>             throw Py.TypeError("function() argument 1 must be code, not " +
>                                code.getType().fastGetName());
>         }
>         ...
>
> I'm confused why its ArgParser defines the arguments in the order of "code", "globals", "name", "argdefs", and "closure".  Shouldn't these be defined in the same order as for the PyFunction constructor (e.g. "globals", "defaults", "code", "doc", "closure_cells")?  Is this a bug in the Jython 2.5.1 code? Any suggestions for how I can get our call to PyFunction() to work with Jython 2.5.1?  Any help is greatly appreciated.

PyFunction became a formally exposed Python type in 2.5. So even though you're importing the org.python.core.PyFunction class, because this is Python code, what you're now getting is its exposed type, which is types.FunctionType. You can't access the original Java class of exposed types for sanity/security reasons.

CPython defines FunctionType's constructor args to be what you see in __new__, here's its docstring:

class function(object)
|  function(code, globals[, name[, argdefs[, closure]]])
|  
|  Create a function object from a code object and a dictionary.
|  The optional name string overrides the name from the code object.
|  The optional argdefs tuple specifies the default argument values.
|  The optional closure tuple supplies the bindings for free variables.

Our PyFunction class constructors never matched the FunctionType in CPython, so we were forced to break compatibility in your code to fix compatibility with CPython. So unfortunately you'll just have to bite the bullet and reorder to the new signature -- I also recommend importing types.FunctionType instead to clarify what's really happening.

--
Philip Jenvey
------------------------------------------------------------------------------
Throughout its 18-year history, RSA Conference consistently attracts the
world's best and brightest in the field, creating opportunities for Conference
attendees to learn about information security's most important issues through
interactions with peers, luminaries and emerging and established companies.
http://p.sf.net/sfu/rsaconf-dev2dev
_______________________________________________
Jython-users mailing list
Jython-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jython-users