jython-users

[Jython-users] Storing code objects

[Jim A. <ji...@tr...>]

Question:

How do I comvert a compiled code object to some kind of bytestream?
'Pickle' can't handle PyCode ("can't pickle ...PyTableCode object"), and
PyCode is not serializable. Am I missing something or is there no way to
do this easily?

Backround:

I am prototyping a program that uses Jython to allow users to design
forms by entering Jython expressions which are evaluated based on a
large set of XML data that is 'scoped' into the PythonInterpreter
beforehand. (Example: 'Customer.Name' or 'for p in Customer.Phones:
print p') We have built a GUI that displays the forms and allows entry
of the expressions, the 'definition' of the form is stored in XML along
with Fonts, justification, and other formatting info. After each
expression is evaluated the result is 'massaged' using various Java
routines to build a complete page.

The previous version used JavaScript (Rhino) but I'm trying to switch it
to Python to make it easier for the form designers.

I'm having a problem, though, with the speed: when interpreting
expressions Jython is half the speed of Rhino: this is critical for this
app.

I timed the compile compared the the 'eval' and found the compile takes
90% of the time:

 PyCode code = __builtin__.compile(evalLine, "<string>", "eval");  //
90%

 PyObject result = python.eval( (PyCode)code );                    //
10% !!

If I could store the compiled 'code objects' within the forms somehow I
think I could increase the speed of the forms 10x, but I can't figure
out a way to get some kind of 'bytestream' of the code object.

-- 
                                 __o
Jim Adrig                      _ \<,_
ji...@tr...          ' `/ ' `
                  ___________ `-'  `-'

Flon's Law:
        There is not now, and never will be, a language in
        which it is the least bit difficult to write bad programs.

Re: [Jython-users] Storing code objects

[<bc...@wo...>]

On Mon, 19 Mar 2001 10:40:40 -0800, you wrote:

>Question:
>
>How do I comvert a compiled code object to some kind of bytestream?
>'Pickle' can't handle PyCode ("can't pickle ...PyTableCode object"), and
>PyCode is not serializable. Am I missing something or is there no way to
>do this easily?

There is no way. Sorry. The code in a PyTableCode is implemented by a
dynamicly created java class and we do not keep the original java
bytecode around. Even if we kept and saved the java bytecode, loading a
page would still have to recreate a java class for each PyCode.

>Backround:
>
>I am prototyping a program that uses Jython to allow users to design
>forms by entering Jython expressions which are evaluated based on a
>large set of XML data that is 'scoped' into the PythonInterpreter
>beforehand. (Example: 'Customer.Name' or 'for p in Customer.Phones:
>print p') We have built a GUI that displays the forms and allows entry
>of the expressions, the 'definition' of the form is stored in XML along
>with Fonts, justification, and other formatting info. After each
>expression is evaluated the result is 'massaged' using various Java
>routines to build a complete page.
>
>The previous version used JavaScript (Rhino) but I'm trying to switch it
>to Python to make it easier for the form designers.
>
>I'm having a problem, though, with the speed: when interpreting
>expressions Jython is half the speed of Rhino: this is critical for this
>app.
>
>I timed the compile compared the the 'eval' and found the compile takes
>90% of the time:
>
> PyCode code = __builtin__.compile(evalLine, "<string>", "eval");  //
>90%
>
> PyObject result = python.eval( (PyCode)code );                    //
>10% !!
>
>If I could store the compiled 'code objects' within the forms somehow I
>think I could increase the speed of the forms 10x, but I can't figure
>out a way to get some kind of 'bytestream' of the code object.

It pains me a lot to say it, but this is one situation where jython's
compile-to-bytecode approach fails to deliver high performance.

Depending on the design of your forms, if you could collect all the
expressions found on a page into one jython source string:

"""
def func001():
   return Customer.Name
def func002():
   for p in Customer.Phones:
      print p
"""

then the slow class creation would only occur once for the page. 

  PyCode code = __builtin__.compile(pagesrc, "<string>", "exec");
  PyStringMap dict = new PyStringMap()
  Py.exec(code, dict, dict)

and the evaluation of each expression would run at full jython speed:

  PyObject result = dict.__getitem__("func001").__call__();


I hope this helps a little.

regards,
finn

Re: [Jython-users] Storing code objects

[Jim A. <ji...@tr...>]

Finn Bock wrote:

> The code in a PyTableCode is implemented by a
> dynamicly created java class and we do not keep the original java
> bytecode around. Even if we kept and saved the java bytecode, loading a
> page would still have to recreate a java class for each PyCode.

Thanks, now I see why that approach was so slow...

> It pains me a lot to say it, but this is one situation where jython's
> compile-to-bytecode approach fails to deliver high performance.
> 
> Depending on the design of your forms, if you could collect all the
> expressions found on a page into one jython source string:
> 
> """
> def func001():
>    return Customer.Name
> def func002():
>    for p in Customer.Phones:
>       print p
> """
> 
> then the slow class creation would only occur once for the page.
> 
>   PyCode code = __builtin__.compile(pagesrc, "<string>", "exec");
>   PyStringMap dict = new PyStringMap()
>   Py.exec(code, dict, dict)
> 
> and the evaluation of each expression would run at full jython speed:
> 
>   PyObject result = dict.__getitem__("func001").__call__();

This is very interesting, I had not considered that approach, since I
did not clearly understand where the slowdown was (creating classes). I
probably could have had something running by now had I thought of it...

But in the meantime, I have been working on a different (more work!)
approach where I build jython source for the entire page, with
'callbacks' to Java routines that do extra work:

  pagePCL = ""
  errorList = []
  ...
  cmdResult = Customer.Total     // User's original expression
  cmdResult = FillFiller.FormatResults(cmdResult,'#,##0.00')
  cmdResult = FillFiller.FillSegments(cmdResult,
((525,2322,478)),false,10,'',error)
  errorList.append(error)
  pagePCL += cmdResult
  ...

This approach requires more restructuring of my code so I have not been
able to benchmark it yet: I'm still working on all the 'callbacks'. Some
of them could be rewritten in Python too but I'm not sure that would buy
me anything.

Then I was going to build a 'cache' (HashMap) of the code-objects for
the forms in a servlet so I can just keep the compiled pages in memory
(I got this idea while checking out 'Python Server Pages':
http://www.ciobriefings.com/psp/ ). There will be hundreds, and
eventually maybe thousands, of forms being run so I'm hoping this will
be OK: a JDOM tree is cached along with an array of pages (although I
may be able to get rid of that with more custom structures later...).

Later, with some more restructuring/rewriting I might be able to build
complete Python forms and compile them to Java classes (and dynamically
load them?), this would save the initial compile (but not much more?).

Thanks again for your reply; do you see any problems with my new
approach ?

-- 
                                 __o
Jim Adrig                      _ \<,_
ji...@tr...          ' `/ ' `
                  ___________ `-'  `-'