From: Kouichi T. <sh...@sf...> - 2003-05-20 09:57:56
|
hi, Summary: Question: Can psyco optimize a python code object? I'm developing a simulation software, E-Cell Simulation Environment Version 3 (ecell.sourceforge.net), and am wondering if Psyco can improve performance of the software. It has an object-oriented simulation model, and allows users to define their own classes in C++ which can be dynamically loaded. It also has a Python frontend API. Recently we are trying to make it possible to define the new classes in Python. Perhaps it will eventually be made possible to define a new class in Python (we will use BPL), but for now we have done a thing like this: Process PythonProcess ( ID ) { Expression "some python code... "; } 'PythonProcess' is a subclass of 'Process' class, and implemented in C++. The 'Expression' string is converted to PyCodeObject by Py_CompileString(), and executed in each simulation step with PyEval_EvalCode(). The expression is often, but not limited to, a differential equation. A preliminary benchmark with a simple simulation model showed that it slows down the simulation about 50 times. Obviously this is a bit slow for real use. And we are trying to speed it up. Now we find psyco, and consider it promising for our use. But a technical problem we have right now is a way to optimize a PyCodeObject structure by psyco. Does psyco have an API function for it? If not, it will be very grateful if psyco provides one, either in C or Python level. best regards, -Kouichi |
From: Armin R. <ar...@tu...> - 2003-05-20 18:33:07
|
Hello Kouichi, On Tue, May 20, 2003 at 06:57:01PM +0900, Kouichi Takahashi wrote: > Summary: Question: Can psyco optimize a python code object? Not directly. The quick answer to this question is that you can use a fun= ction object instead of a code object; for example, from Python, instead of co =3D compile("...expr...") use f =3D eval("lambda x,y: ...expr...") This is a general trick, by the way: bytecode objects generated by the fo= rmer=20 are slower than the ones generated by the latter, because the 'lambda'=20 expression gives the compiler more information about which variables are=20 locals and which are globals. The drawback is that calling a function has= a=20 larger overhead than starting the execution of a code object. So the quick-and-dirty trick for Psyco is to evaluate the expression "psyco.proxy(lambda x,y: ...expr...)", which gives a callable object that= you can use with e.g. PyObject_Call(). Be aware however that the overhead of calling a Psyco proxy is much large= r than even calling a Python function. In other words, it works well only f= or Python code that runs for some time (e.g. a loop). If, for example, you w= ant to compute a small expression for all values of x in some range, consider moving the loop into Python code --- in other words, build the string psyco.proxy(lambda xlist: [...expr... for x in xlist]) then evaluate it with eval() or, from C, PyRun_String(), and finally call= it (with PyObject_Call() from C). Psyco will compile it into a reasonably fa= st loop. A bient=F4t, Armin. |
From: Kouichi T. <sh...@sf...> - 2003-05-21 16:44:49
|
Thank you Armin, > Not directly. The quick answer to this question is that you can use a function > object instead of a code object; for example, from Python, instead of > > co = compile("...expr...") > > use > > f = eval("lambda x,y: ...expr...") Neat trick. Actually a typical code we want to execute can be in some cases very trivial, and called repeatedly during the simulation (say, one million times to proceed the simulation only 1000 seconds). An example is: k = 3e10; m = 1.0; C0.Value * S0.Concentration / ( S0.Concentration + m ) S0 is a C++ object wrapped by BPL, and property accesses (such as S0.Value) call simple C++ methods. I have tested with some trivial python codes and found the lambda approach yields almost twice better performance if the code is trivial. But psyco.bind() slowed down the code about 10-20%, probably because of the overhead. Of course using weave or PyRex can be some other options, but I don't want to force our users to learn another language. Python is very clean, and most users are already familiar with it. > So the quick-and-dirty trick for Psyco is to evaluate the expression > "psyco.proxy(lambda x,y: ...expr...)", which gives a callable object that you > can use with e.g. PyObject_Call(). or a possible tweaking is PyFunction_GET_CODE() and then PyEval_EvalCode()? We don't need to give arguments to the function. > Be aware however that the overhead of calling a Psyco proxy is much larger > than even calling a Python function. In other words, it works well only for > Python code that runs for some time (e.g. a loop). If, for example, you want > to compute a small expression for all values of x in some range, consider > moving the loop into Python code --- in other words, build the string > > psyco.proxy(lambda xlist: [...expr... for x in xlist]) This is not the case for us, unfortunately. It is called just once in a simulation iteration. -Kouichi |