From: Leo U. <leo...@ya...> - 2006-12-11 21:52:08
|
well, things have progressed to the point where I can start up the modified jython and actually get to the point of having a prompt show up and being able to interact with it. This means instead of this type of lovely stack: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:174) at org.python.core.PyMethod.__call__(PyMethod.java:93) at org.python.core.PyObject.__call__(PyObject.java:582) we should see something like: at TestOtestO634.jcall(Unknown Source) at org.python.core.PyJavaMethodProxy.__call__(PyJavaMethodProxy.java:411) at org.python.core.PyObject.__call__(PyObject.java:628) There are numerous problems left to investigate and find a solution for still. But I like the fact that I can import a java class and call toString on it without any reflective access to it. One hurdle is to be able to manage the object being called. __call__ seems assume various things. Sometimes a __call__ path will get a reference to a "self" object, sometimes not. Its the "not" that needs to be worked on. leouser ____________________________________________________________________________________ Want to start your own business? Learn how on Yahoo! Small Business. http://smallbusiness.yahoo.com/r-index |
From: Leo U. <leo...@ya...> - 2006-12-11 23:45:00
|
yup, the overhead saved appears to be good. From running this code: >>> def zoo(): ... s = jl.String("VOODOO") ... x = st() ... for z in xrange(300000): ... s.toString() ... x2 = st() ... print (x2 - x) ... in a modified jython3005 and an unmodified version on Java 6: unmodified: >>> for z in xrange(20): ... zoo() ... 364 368 370 364 366 367 365 374 375 372 380 372 373 370 369 367 368 366 370 367 MODIFIED: >>> for z in xrange(20): ... zoo() ... 285 292 291 293 289 288 288 288 296 289 297 290 295 292 289 288 292 294 287 287 any savings should come from the fact that we are swapping reflection for invokevirtual. I believe xrange is getting the PyJavaMethodProxy treatment as well. Im not sure if its needed, will have to look if reflection is used to invoke it or if its just a plain ol method call. Im not sure if there is some function we can use to gauge how much less expensive one version is from the other. Just increasing the loop iteration to 3000000 changes the savings to about 1s. leouser ____________________________________________________________________________________ Need a quick answer? Get one in minutes from people who know. Ask your question on www.Answers.yahoo.com |
From: Leo U. <leo...@ya...> - 2006-12-12 18:47:57
|
Hmmm, it may be faster but it also appears to add alot of fat at runtime. :( From looking at the difference between the two's footprints, it looks like the current piece of code increases the size by 50%. That's just with a "jython" started up and no scripts running. This probably could be reduced some if the cache was more efficient. I believe in these cases: public class Test{ public void test(); } public class Test2 extends Test{} there is going to be a test() class generated for each class when 1 would suffice. Another technique could be to delay class creation until needed. As it stands now, each new type builds all of the classes it needs. leouser --- Leo User <leo...@ya...> wrote: > yup, the overhead saved appears to be good. From > running this code: > >>> def zoo(): > ... s = jl.String("VOODOO") > ... x = st() > ... for z in xrange(300000): > ... s.toString() > ... x2 = st() > ... print (x2 - x) > ... > > in a modified jython3005 and an unmodified version > on > Java 6: > unmodified: > >>> for z in xrange(20): > ... zoo() > ... > 364 > 368 > 370 > 364 > 366 > 367 > 365 > 374 > 375 > 372 > 380 > 372 > 373 > 370 > 369 > 367 > 368 > 366 > 370 > 367 > > MODIFIED: > >>> for z in xrange(20): > ... zoo() > ... > 285 > 292 > 291 > 293 > 289 > 288 > 288 > 288 > 296 > 289 > 297 > 290 > 295 > 292 > 289 > 288 > 292 > 294 > 287 > 287 > > any savings should come from the fact that we are > swapping reflection for invokevirtual. I believe > xrange is getting the PyJavaMethodProxy treatment as > well. Im not sure if its needed, will have to look > if > reflection is used to invoke it or if its just a > plain > ol method call. > > Im not sure if there is some function we can use to > gauge how much less expensive one version is from > the > other. Just increasing the loop iteration to > 3000000 > changes the savings to about 1s. > > leouser > > > > ____________________________________________________________________________________ > Need a quick answer? Get one in minutes from people > who know. > Ask your question on www.Answers.yahoo.com > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get > the chance to share your > opinions on IT & business topics through brief > surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > ____________________________________________________________________________________ Do you Yahoo!? Everyone is raving about the all-new Yahoo! Mail beta. http://new.mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2006-12-12 19:48:39
|
Hmmm, I should learn to look at my top numbers better. Actually the fat isn't that big for the startup. Its about 2M instead of 15M as I was somehow thinking. That's not too terrible. I put in some deferred class generation code and it appears to keep the startup sizes at about the same size. With the deferred code in place the system generates 13 PyJavaMethodProxy classes. Without it, the number is around 500. leouser --- Leo User <leo...@ya...> wrote: > Hmmm, it may be faster but it also appears to add > alot > of fat at runtime. :( From looking at the > difference > between the two's footprints, it looks like the > current piece of code increases the size by 50%. > That's just with a "jython" started up and no > scripts > running. This probably could be reduced some if the > cache was more efficient. I believe in these cases: > public class Test{ > public void test(); > } > > public class Test2 extends Test{} > > there is going to be a test() class generated for > each > class when 1 would suffice. > > Another technique could be to delay class creation > until needed. As it stands now, each new type > builds > all of the classes it needs. > > leouser > > > --- Leo User <leo...@ya...> wrote: > > > yup, the overhead saved appears to be good. From > > running this code: > > >>> def zoo(): > > ... s = jl.String("VOODOO") > > ... x = st() > > ... for z in xrange(300000): > > ... s.toString() > > ... x2 = st() > > ... print (x2 - x) > > ... > > > > in a modified jython3005 and an unmodified version > > on > > Java 6: > > unmodified: > > >>> for z in xrange(20): > > ... zoo() > > ... > > 364 > > 368 > > 370 > > 364 > > 366 > > 367 > > 365 > > 374 > > 375 > > 372 > > 380 > > 372 > > 373 > > 370 > > 369 > > 367 > > 368 > > 366 > > 370 > > 367 > > > > MODIFIED: > > >>> for z in xrange(20): > > ... zoo() > > ... > > 285 > > 292 > > 291 > > 293 > > 289 > > 288 > > 288 > > 288 > > 296 > > 289 > > 297 > > 290 > > 295 > > 292 > > 289 > > 288 > > 292 > > 294 > > 287 > > 287 > > > > any savings should come from the fact that we are > > swapping reflection for invokevirtual. I believe > > xrange is getting the PyJavaMethodProxy treatment > as > > well. Im not sure if its needed, will have to > look > > if > > reflection is used to invoke it or if its just a > > plain > > ol method call. > > > > Im not sure if there is some function we can use > to > > gauge how much less expensive one version is from > > the > > other. Just increasing the loop iteration to > > 3000000 > > changes the savings to about 1s. > > > > leouser > > > > > > > > > ____________________________________________________________________________________ > > Need a quick answer? Get one in minutes from > people > > who know. > > Ask your question on www.Answers.yahoo.com > > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of > IT > > Join SourceForge.net's Techsay panel and you'll > get > > the chance to share your > > opinions on IT & business topics through brief > > surveys - and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Jython-dev mailing list > > Jyt...@li... > > > https://lists.sourceforge.net/lists/listinfo/jython-dev > > > > > > > ____________________________________________________________________________________ > Do you Yahoo!? > Everyone is raving about the all-new Yahoo! Mail > beta. > http://new.mail.yahoo.com > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get > the chance to share your > opinions on IT & business topics through brief > surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > ____________________________________________________________________________________ Do you Yahoo!? Everyone is raving about the all-new Yahoo! Mail beta. http://new.mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2006-12-14 17:05:30
|
Some minor progress with the PyJavaMethodProxy. Ive developed it to the point where it crashes with the exact same stack as the stack printed out for an unmodified jython3005. Not that its good that its crashing, it is good that it is the same crash. A key piece appears to be to be able to just forward the parameters if the param list matches what Id expect for a python call: method(PyObject[] args, String[] kwords) method(PyObject[] args) this helps in that there doesn't need to be any unpacking or packing of the arguments, or conversions for that matter. I am a little worried about mixed overloads: method(PyObject[] args) method(int i) I suppose if the args carries one argument and it can be converted to an Integer, the method invocations should be method(int i), since its a more exact match(or is it?). I hope that there aren't diabolical situations like: method(PyObject[] args, String[] kwords) method(PyObject[] args) I guess if there is, maybe the answer would be to see if kwords length is 0 and if not pass it to the first. If it isn't pass it to the second. The one time Ive ran into this is when the method in question is the __call__ method. In this case it may suffice to just provide a thin wrapper over the __call__ methods. leouser ____________________________________________________________________________________ Cheap talk? Check out Yahoo! Messenger's low PC-to-Phone call rates. http://voice.yahoo.com |
From: Samuele P. <ped...@op...> - 2006-12-14 17:53:41
|
Leo User wrote: > Some minor progress with the PyJavaMethodProxy. Ive > developed it to the point where it crashes with the > exact same stack as the stack printed out for an > unmodified jython3005. Not that its good that its > crashing, it is good that it is the same crash. > > A key piece appears to be to be able to just forward > the parameters if the param list matches what Id > expect for a python call: > method(PyObject[] args, String[] kwords) > method(PyObject[] args) > > this helps in that there doesn't need to be any > unpacking or packing of the arguments, or conversions > for that matter. I am a little worried about mixed > overloads: > method(PyObject[] args) > method(int i) > most of the complexity of the original PyReflectedFunction and related classes is about properly resolving and converting in the presence of overloaded Java methods. This is all necessary in the general case. The necessary logic is involved, and its current incarnation still buggy (there's a prototype of better rules in the sandbox), also is not flexible enough to do the right thing in case we had Python bool and in the presence of int/boolean overloading. This all needs to be cleaned up and fixed before thinking about shortcuts. Notice also that Jython is supposed to work to some extent even in the presence of a security manager, this rules out approaches purely based on generating classes at runtime. This is one of the areas for which jythonc was most useful, and jythonc code was still able to cope with arbitrary java classes at runtime because of the runtime generally using reflection. An adaptive combination of approaches would probably be the best solution. |
From: Leo U. <leo...@ya...> - 2006-12-14 18:17:19
|
--- Samuele Pedroni <ped...@op...> wrote: > Leo User wrote: > > Some minor progress with the PyJavaMethodProxy. > Ive > > developed it to the point where it crashes with > the > > exact same stack as the stack printed out for an > > unmodified jython3005. Not that its good that its > > crashing, it is good that it is the same crash. > > > > A key piece appears to be to be able to just > forward > > the parameters if the param list matches what Id > > expect for a python call: > > method(PyObject[] args, String[] kwords) > > method(PyObject[] args) > > > > this helps in that there doesn't need to be any > > unpacking or packing of the arguments, or > conversions > > for that matter. I am a little worried about > mixed > > overloads: > > method(PyObject[] args) > > method(int i) > > > most of the complexity of the original > PyReflectedFunction and related > classes is about properly resolving and converting > in the presence of > overloaded Java methods. This is all necessary in > the general case. Id be surprised if there ever was a true way to deal with ambiguous situations well. Maybe there is. Ive got a couple of ideas to try out to make the situation simpler further still. > > The necessary logic is involved, and its current > incarnation still buggy > (there's a prototype of better rules in the > sandbox), also is not > flexible enough to do the right thing in case we had > Python bool and in > the presence of int/boolean overloading. The aim in generation is to do most of the precomputation at generation time. And if what it can't do then, try to do as simply as possible during invocation. > > This all needs to be cleaned up and fixed before > thinking about shortcuts. > > Notice also that Jython is supposed to work to some > extent even in the > presence of a security manager, this rules out > approaches purely based > on generating classes at runtime. This is one of the > areas for which > jythonc was most useful, and jythonc code was still > able to cope with > arbitrary java classes at runtime because of the > runtime generally using > reflection. > Im not sure why this is a negative. Jython already is defining classes during runtime, so it isn't a foreign event that is being introduced by generating proxies. > An adaptive combination of approaches would probably > be the best solution. > Ive pondered using some form of method invocation monitoring to decide if the proxy should be reflective or plain old byte code. On the other hand, cranking method calls as close to the method as possible in any situation seems nice. What will be interesting to see is if the invokedynamic instruction could simplify this stuff even further. Ill have to reread the what's out there, but it seemed like alot of cast checking could be dispensed with. leouser ____________________________________________________________________________________ Have a burning question? Go to www.Answers.yahoo.com and get answers from real people who know. |
From: Leo U. <leo...@ya...> - 2006-12-14 19:12:33
|
Hmm, it looks like some elementary changes to the caching system can speed things up even further. From running my TestLoopTime.py test, Im seeing a drop of about 300 more milliseconds from execution. UNMODIFIED TIMES: 2087 2023 2015 2022 2017 MODIFIED TIMES: 978 915 891 899 897 These were in the 1000 - 1200 range before modifying. Im a little surprised to see the jump. Im not sure why its happening, is it because: a. invokevirtual is more efficient when invoking a method against the highest class that declares an overriden method? b. The cache is that much faster(doubtful)? c. Fewer classes because of better cache management(doubtful here as well)? leouser ____________________________________________________________________________________ Any questions? Get answers on any topic at www.Answers.yahoo.com. Try it now. |
From: Leo U. <leo...@ya...> - 2006-12-15 16:19:54
|
Some more progress here, Im able to get the jyleo gui up and running. Class generation from startup to gui showing looks like its in the 300 range. The odd thing is that it looks like the modified jython version doesn't grow in memory consumption as quickly as the unmodified version. Im going to have to do some more fiddling around to see if there is a trend. But I was watching the, what I think, is the memory used and the unmodified version had no problem add megabytes to the total count while the modified one added less with more operations. Maybe the unmodified version just adds alot of garbage to the system? Well, that's the first peek at the thing running an app. Its kind of fun to see the classes get spit out on the fly as needed. leouser __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2006-12-16 00:41:56
|
Hmm, an intriguing problem has emerged: SuperClassofCurrentClass.invokeSuperMethod(self) which means that the current class if it invokes a super method. The code does not deal with this situation at present. Im not sure what the answer is yet, but it sure is nasty. :D Im thinking that the invocation is going to have to happen in the non-proxy generated code. It would be nice if in the byte code this: ClassOfSomething.aMethod(self) just turned into the "invokespecial" opcode. Im not sure if this idea is doable without a major alteration. So will have to wait and see. Ill have to look at the byte code generation more closely. I may have tried doing an "invokespecial" in a generated PyJavaMethodProxy but I believe this op is illegal unless the current class is a subclass of invokee. leouser __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2006-12-17 18:29:27
|
I think the answer may that if the code finds itself in a situation where it appears a super call is needed, to create a proxy for the super__* method and invoke that. From observing the code generated by a jython class that overrides a Java class, the overriden method has a peer method that starts with "super__". The PyReflectedFunction class appears to take this route if it meets certain criteria. An elementary patch appears to show this as a feasible strategy as well. leouser --- Leo User <leo...@ya...> wrote: > Hmm, an intriguing problem has emerged: > SuperClassofCurrentClass.invokeSuperMethod(self) > > which means that the current class if it invokes a > super method. The code does not deal with this > situation at present. Im not sure what the answer > is > yet, but it sure is nasty. :D Im thinking that the > invocation is going to have to happen in the > non-proxy > generated code. It would be nice if in the byte > code > this: > ClassOfSomething.aMethod(self) > > just turned into the "invokespecial" opcode. Im not > sure if this idea is doable without a major > alteration. So will have to wait and see. > Ill > have to look at the byte code generation more > closely. > > I may have tried doing an "invokespecial" in a > generated PyJavaMethodProxy but I believe this op is > illegal unless the current class is a subclass of > invokee. > > leouser > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam > protection around > http://mail.yahoo.com > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get > the chance to share your > opinions on IT & business topics through brief > surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2006-12-17 23:21:14
|
Yup, this appears to be the/a path to doing this. Interestingly, it seems to not take that much time to define and instantiate a proxy. From observing a jyleo startup sequence, it ranged normally from 1-7 milliseconds to go through the process. There was an occasional oddball, and it would be interesting to see why they were odd but for the most part it seems relatively cheap timewise. Though I did notice a trend with things moving towards the 7 end at the end. At the start it was more in the 0-2 area. Maybe there is something a little linear about the cache? leouser --- Leo User <leo...@ya...> wrote: > I think the answer may that if the code finds itself > in a situation where it appears a super call is > needed, to create a proxy for the super__* method > and > invoke that. From observing the code generated by a > jython class that overrides a Java class, the > overriden method has a peer method that starts with > "super__". The PyReflectedFunction class appears to > take this route if it meets certain criteria. An > elementary patch appears to show this as a feasible > strategy as well. > > leouser > > > --- Leo User <leo...@ya...> wrote: > > > Hmm, an intriguing problem has emerged: > > SuperClassofCurrentClass.invokeSuperMethod(self) > > > > which means that the current class if it invokes a > > super method. The code does not deal with this > > situation at present. Im not sure what the answer > > is > > yet, but it sure is nasty. :D Im thinking that > the > > invocation is going to have to happen in the > > non-proxy > > generated code. It would be nice if in the byte > > code > > this: > > ClassOfSomething.aMethod(self) > > > > just turned into the "invokespecial" opcode. Im > not > > sure if this idea is doable without a major > > alteration. So will have to wait and see. > > Ill > > have to look at the byte code generation more > > closely. > > > > I may have tried doing an "invokespecial" in a > > generated PyJavaMethodProxy but I believe this op > is > > illegal unless the current class is a subclass of > > invokee. > > > > leouser > > > > __________________________________________________ > > Do You Yahoo!? > > Tired of spam? Yahoo! Mail has the best spam > > protection around > > http://mail.yahoo.com > > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of > IT > > Join SourceForge.net's Techsay panel and you'll > get > > the chance to share your > > opinions on IT & business topics through brief > > surveys - and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Jython-dev mailing list > > Jyt...@li... > > > https://lists.sourceforge.net/lists/listinfo/jython-dev > > > > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam > protection around > http://mail.yahoo.com > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get > the chance to share your > opinions on IT & business topics through brief > surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2006-12-18 14:53:55
|
Another target for generation may be the PyGetSetDescr. It looks like a simple class whose purpose is to get and set the value of something on a class. Reflection is used for the get_meth and set_meth, one Method apiece. So it shouldn't be that much of a task to replace the reflective call with a simple method call. There shouldn't be any ambiguity in the situation since the Method is just a point in the code. Im not sure about PyReflectedConstructor yet, there is alot more to it than invoking a super call. Will have to see. :) leouser __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2006-12-21 21:11:58
|
Hmm, it appears that gutting PyStringMap so that it uses a HashMap internally also has an effect on the timings: 821 751 748 742 739 742 738 733 734 732 this assumes of course that my reimplementation of PyStringMap is correct. It appears that it may shave off another 250 milliseconds from the test. leouser --- Leo User <leo...@ya...> wrote: > Hmm, it looks like some elementary changes to the > caching system can speed things up even further. > From > running my TestLoopTime.py test, Im seeing a drop of > about 300 more milliseconds from execution. > > UNMODIFIED TIMES: > 2087 > 2023 > 2015 > 2022 > 2017 > > MODIFIED TIMES: > 978 > 915 > 891 > 899 > 897 > > These were in the 1000 - 1200 range before > modifying. > Im a little surprised to see the jump. Im not sure > why its happening, is it because: > a. invokevirtual is more efficient when invoking a > method against the highest class that declares an > overriden method? > > b. The cache is that much faster(doubtful)? > > c. Fewer classes because of better cache > management(doubtful here as well)? > > leouser > > > > > > ____________________________________________________________________________________ > Any questions? Get answers on any topic at > www.Answers.yahoo.com. Try it now. > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get > the chance to share your > opinions on IT & business topics through brief > surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2007-01-02 17:07:13
|
>From the looks of things(profiling this test): 12.1% 69 + 0 java.util.HashMap.get 11.0% 63 + 0 org.python.core.PyType.fromClass 7.3% 39 + 3 org.python.core.Py.java2py 20.5% 0 + 117 java.lang.Class.isInstance 3.8% 0 + 22 java.lang.Class.isPrimitive The biggest time consuming beast appears to be isInstance. HashMap.get would probably be Hashtable.get, considering that Im running a modified PyDictionary. PyType.fromClass eats up some time. I don't know if its possible, but if the usage of isInstance could be swapped with something that isn't a native method maybe we could get rid of its large consumption of the runtime. I see that PyInstance uses isInstance a couple of times in __tojava__, which may account for its bloated representation here. What's nice is seeing that the PyJavaMethodProxy instances that have been generated have had their java call methods converted to native methods. I suspected that would be the case, given their very simple nature. Actually most of the PyJavaMethodProxy appears to become compiled. onward, leouser --- Leo User <leo...@ya...> wrote: > Hmm, > > it appears that gutting PyStringMap so that it uses > a > HashMap internally also has an effect on the > timings: > 821 > 751 > 748 > 742 > 739 > 742 > 738 > 733 > 734 > 732 > > this assumes of course that my reimplementation of > PyStringMap is correct. It appears that it may > shave > off another 250 milliseconds from the test. > > leouser > > > --- Leo User <leo...@ya...> wrote: > > > Hmm, it looks like some elementary changes to the > > caching system can speed things up even further. > > From > > running my TestLoopTime.py test, Im seeing a drop > of > > about 300 more milliseconds from execution. > > > > UNMODIFIED TIMES: > > 2087 > > 2023 > > 2015 > > 2022 > > 2017 > > > > MODIFIED TIMES: > > 978 > > 915 > > 891 > > 899 > > 897 > > > > These were in the 1000 - 1200 range before > > modifying. > > Im a little surprised to see the jump. Im not > sure > > why its happening, is it because: > > a. invokevirtual is more efficient when invoking a > > method against the highest class that declares an > > overriden method? > > > > b. The cache is that much faster(doubtful)? > > > > c. Fewer classes because of better cache > > management(doubtful here as well)? > > > > leouser > > > > > > > > > > > > > ____________________________________________________________________________________ > > Any questions? Get answers on any topic at > > www.Answers.yahoo.com. Try it now. > > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of > IT > > Join SourceForge.net's Techsay panel and you'll > get > > the chance to share your > > opinions on IT & business topics through brief > > surveys - and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Jython-dev mailing list > > Jyt...@li... > > > https://lists.sourceforge.net/lists/listinfo/jython-dev > > > > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam > protection around > http://mail.yahoo.com > > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2007-01-02 18:48:49
|
Well, it appears removing the calls to isInstance in PyInstance to a homegrown method where there is a cache of Classes per class that represents the "Assignable set" cuts another 100 milliseconds from the test: 732 667 648 642 641 652 641 644 643 649 641 Im not sure how much this technique can be supported, essentially we end up keeping a Class[] in memory for each conversion tried. Maybe it will prove a worthwhile way of doing it. Converting to this changes the big consumers somewhat: 14.5% 69 + 0 java.util.HashMap.get 12.8% 53 + 8 org.python.core.Py.java2py 10.1% 48 + 0 org.python.core.PyType.fromClass 9.7% 46 + 0 org.python.core.PyInstance.isInstance The PyInstance.isInstance is the new homegrown method. Suddenly it appears that HashMap.get is the leader. I have to wonder why PyType.fromClass is consuming so much. If its a one time event per class, maybe we won't worry about it now but if its something that is always happening, we will have to see about cutting it down. Py.java2py as well, not sure about that method. Ive looked at that before, its an ugly thing. leouser --- Leo User <leo...@ya...> wrote: > >From the looks of things(profiling this test): > 12.1% 69 + 0 java.util.HashMap.get > 11.0% 63 + 0 > org.python.core.PyType.fromClass > 7.3% 39 + 3 org.python.core.Py.java2py > 20.5% 0 + 117 java.lang.Class.isInstance > 3.8% 0 + 22 java.lang.Class.isPrimitive > > > The biggest time consuming beast appears to be > isInstance. HashMap.get would probably be > Hashtable.get, considering that Im running a > modified > PyDictionary. PyType.fromClass eats up some time. > I > don't know if its possible, but if the usage of > isInstance could be swapped with something that > isn't > a native method maybe we could get rid of its large > consumption of the runtime. I see that PyInstance > uses isInstance a couple of times in __tojava__, > which may account for its bloated representation > here. > > What's nice is seeing that the PyJavaMethodProxy > instances that have been generated have had their > java > call methods converted to native methods. I > suspected > that would be the case, given their very simple > nature. Actually most of the PyJavaMethodProxy > appears to become compiled. > > onward, > leouser > > > > > > --- Leo User <leo...@ya...> wrote: > > > Hmm, > > > > it appears that gutting PyStringMap so that it > uses > > a > > HashMap internally also has an effect on the > > timings: > > 821 > > 751 > > 748 > > 742 > > 739 > > 742 > > 738 > > 733 > > 734 > > 732 > > > > this assumes of course that my reimplementation of > > PyStringMap is correct. It appears that it may > > shave > > off another 250 milliseconds from the test. > > > > leouser > > > > > > --- Leo User <leo...@ya...> wrote: > > > > > Hmm, it looks like some elementary changes to > the > > > caching system can speed things up even further. > > > > From > > > running my TestLoopTime.py test, Im seeing a > drop > > of > > > about 300 more milliseconds from execution. > > > > > > UNMODIFIED TIMES: > > > 2087 > > > 2023 > > > 2015 > > > 2022 > > > 2017 > > > > > > MODIFIED TIMES: > > > 978 > > > 915 > > > 891 > > > 899 > > > 897 > > > > > > These were in the 1000 - 1200 range before > > > modifying. > > > Im a little surprised to see the jump. Im not > > sure > > > why its happening, is it because: > > > a. invokevirtual is more efficient when invoking > a > > > method against the highest class that declares > an > > > overriden method? > > > > > > b. The cache is that much faster(doubtful)? > > > > > > c. Fewer classes because of better cache > > > management(doubtful here as well)? > > > > > > leouser > > > > > > > > > > > > > > > > > > > > > ____________________________________________________________________________________ > > > Any questions? Get answers on any topic at > > > www.Answers.yahoo.com. Try it now. > > > > > > > > > ------------------------------------------------------------------------- > > > Take Surveys. Earn Cash. Influence the Future of > > IT > > > Join SourceForge.net's Techsay panel and you'll > > get > > > the chance to share your > > > opinions on IT & business topics through brief > > > surveys - and earn cash > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > _______________________________________________ > > > Jython-dev mailing list > > > Jyt...@li... > > > > > > https://lists.sourceforge.net/lists/listinfo/jython-dev > > > > > > > > > __________________________________________________ > > Do You Yahoo!? > > Tired of spam? Yahoo! Mail has the best spam > > protection around > > http://mail.yahoo.com > > > > > > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam > protection around > http://mail.yahoo.com > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get > the chance to share your > opinions on IT & business topics through brief > surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2007-01-02 19:25:34
|
The fromClass call was being invoked each time a special class in the PyJavaMethodProxy was being instantiated. Cutting that out, stops that method from being invoked and reduces the times, again, to: 641 561 551 556 550 549 558 539 541 543 546 so roughly another 100 milliseconds has been shaved from the test. The new time monsters appear to be: 18.4% 79 + 0 java.util.HashMap.get 10.2% 40 + 4 org.python.core.Py.java2py 8.8% 24 + 14 org.python.core.PyJavaMethodProxy.convert 6.3% 5 + 22 org.python.core.PyClass.lookupGivingClass 6.0% 26 + 0 org.python.core.PyJavaMethodProxy.__call__ 5.3% 23 + 0 org.python.core.PyInstance.isInstance 4.9% 21 + 0 org.python.core.PyInstance.__findattr__ 4.2% 18 + 0 org.python.core.PyObject.__get__ I guess java2py needs to be investigated next. Im surprised to see the 500ish millisecond range being hit. This means we only need to shave off another 500 milliseconds and we will be equaling Java in terms of method performance!(ahem, cough cough) :D Beyond that group of time consumers, Im not sure what is next. Everything else appears to be marginal( < 1%). leouser --- Leo User <leo...@ya...> wrote: > Well, > > it appears removing the calls to isInstance in > PyInstance to a homegrown method where there is a > cache of Classes per class that represents the > "Assignable set" cuts another 100 milliseconds from > the test: > 732 > 667 > 648 > 642 > 641 > 652 > 641 > 644 > 643 > 649 > 641 > > Im not sure how much this technique can be > supported, > essentially we end up keeping a Class[] in memory > for > each conversion tried. Maybe it will prove a > worthwhile way of doing it. Converting to this > changes the big consumers somewhat: > 14.5% 69 + 0 java.util.HashMap.get > 12.8% 53 + 8 org.python.core.Py.java2py > 10.1% 48 + 0 > org.python.core.PyType.fromClass > 9.7% 46 + 0 > org.python.core.PyInstance.isInstance > > The PyInstance.isInstance is the new homegrown > method. > Suddenly it appears that HashMap.get is the leader. > > I have to wonder why PyType.fromClass is consuming > so > much. If its a one time event per class, maybe we > won't worry about it now but if its something that > is > always happening, we will have to see about cutting > it > down. Py.java2py as well, not sure about that > method. > > > > > > --- Leo User <leo...@ya...> wrote: > > > > > > > Hmm, it looks like some elementary changes to > > the > > > > caching system can speed things up even > further. > > > > > > From > > > > running my TestLoopTime.py test, Im seeing a > > drop > > > of > > > > about 300 more milliseconds from execution. > > > > > > > > UNMODIFIED TIMES: > > > > 2087 > > > > 2023 > > > > 2015 > > > > 2022 > > > > 2017 __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2007-01-02 22:06:35
|
Some more improvements: 579 497 485 484 483 485 495 486 484 485 483 What's happening here is that in the PyJavaMethodProxy Ive concoted a specialised version of java2py. This one instead of checking the argument to see what it is already has an argument number assigned to it. This feeds into a switch statement that converts it like, Py.java2py does. The assigned number is created when the PyJavaMethodProxy subclass is synthesized. This roughly cuts 50-60 more milliseconds from the test. The new time eaters are: 13.4% 56 + 0 org.python.core.PyInstance.isInstance 11.0% 46 + 0 java.util.HashMap.get 9.8% 25 + 16 org.python.core.PyClass.lookupGivingClass 7.4% 31 + 0 org.python.core.Py.java2py 6.9% 20 + 9 org.python.core.PyJavaMethodProxy.convert 5.3% 22 + 0 org.python.core.PyJavaMethodProxy.__call__ 5.3% 22 + 0 org.python.core.PyObject.__get__ 5.0% 21 + 0 org.python.core.PyJavaClass.lookupGivingClass Maybe these lookupGivingClass methods will be worthwhile exploring. leouser --- Leo User <leo...@ya...> wrote: > The fromClass call was being invoked each time a > special class in the PyJavaMethodProxy was being > instantiated. Cutting that out, stops that method > from being invoked and reduces the times, again, to: > 641 > 561 > 551 > 556 > 550 > 549 > 558 > 539 > 541 > 543 > 546 > > > > > Hmm, it looks like some elementary changes > to > > > the > > > > > caching system can speed things up even > > further. > > > > > > > > From > > > > > running my TestLoopTime.py test, Im seeing a > > > drop > > > > of > > > > > about 300 more milliseconds from execution. > > > > > > > > > > UNMODIFIED TIMES: > > > > > 2087 > > > > > 2023 > > > > > 2015 > > > > > 2022 > > > > > 2017 > > > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam > protection around > http://mail.yahoo.com > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get > the chance to share your > opinions on IT & business topics through brief > surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2007-01-03 17:17:57
|
Another technique cuts things down further: 500 449 429 428 430 432 435 444 429 429 438 This would be swapping out the assignable set check with a simple class generation scheme for object of type InstanceOf. The generated classes are very simple, they just do this: return o instanceof YourClassHere Which is what the code is crying for. Ive seen this take the timings down to 414. Of course, again, this raises the question of expense. How much more memory are we going to eat by having InstanceOfs floating around. Thankfully, there will just be one per class. From looking at the core package there are other possible candidate classes that could use this, so maybe it would pay for itself beyond the PyInstance. profile leaders: 15.3% 57 + 0 java.util.HashMap.get 9.1% 34 + 0 org.python.core.PyInstance.__findattr__ 8.1% 25 + 5 org.python.core.PyJavaMethodProxy.convert 8.1% 30 + 0 org.python.core.PyInstance.getInstanceOf leouser --- Leo User <leo...@ya...> wrote: > Some more improvements: > 579 > 497 > 485 > 484 > 483 > 485 > 495 > 486 > 484 > 485 > 483 > > What's happening here is that in the > PyJavaMethodProxy > Ive concoted a specialised version of java2py. This > one instead of checking the argument to see what it > is > already has an argument number assigned to it. This > feeds into a switch statement that converts it like, > Py.java2py does. The assigned number is created > when > the PyJavaMethodProxy subclass is synthesized. This > roughly cuts 50-60 more milliseconds from the test. > > The new time eaters are: > 13.4% 56 + 0 > org.python.core.PyInstance.isInstance > 11.0% 46 + 0 java.util.HashMap.get > 9.8% 25 + 16 > org.python.core.PyClass.lookupGivingClass > 7.4% 31 + 0 org.python.core.Py.java2py > 6.9% 20 + 9 > org.python.core.PyJavaMethodProxy.convert > 5.3% 22 + 0 > org.python.core.PyJavaMethodProxy.__call__ > 5.3% 22 + 0 > org.python.core.PyObject.__get__ > 5.0% 21 + 0 > org.python.core.PyJavaClass.lookupGivingClass > > > Maybe these lookupGivingClass methods will be > worthwhile exploring. > > leouser > > > --- Leo User <leo...@ya...> wrote: > > > The fromClass call was being invoked each time a > > special class in the PyJavaMethodProxy was being > > instantiated. Cutting that out, stops that method > > from being invoked and reduces the times, again, > to: > > 641 > > 561 > > 551 > > 556 > > 550 > > 549 > > 558 > > 539 > > 541 > > 543 > > 546 > > > > > > Hmm, it looks like some elementary changes > > to > > > > the > > > > > > caching system can speed things up even > > > further. > > > > > > > > > > From > > > > > > running my TestLoopTime.py test, Im seeing > a > > > > drop > > > > > of > > > > > > about 300 more milliseconds from > execution. > > > > > > > > > > > > UNMODIFIED TIMES: > > > > > > 2087 > > > > > > 2023 > > > > > > 2015 > > > > > > 2022 > > > > > > 2017 > > > > > > > > __________________________________________________ > > Do You Yahoo!? > > Tired of spam? Yahoo! Mail has the best spam > > protection around > > http://mail.yahoo.com > > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of > IT > > Join SourceForge.net's Techsay panel and you'll > get > > the chance to share your > > opinions on IT & business topics through brief > > surveys - and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Jython-dev mailing list > > Jyt...@li... > > > https://lists.sourceforge.net/lists/listinfo/jython-dev > > > > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam > protection around > http://mail.yahoo.com > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get > the chance to share your > opinions on IT & business topics through brief > surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2007-01-03 17:38:33
|
The list of other classes that could be helped by InstaceOf are: PyInstance PyObject PyString PyArray each one of these uses the dreaded Class.isInstance call. Im not sure if InstanceOf can be a drop in replacement for isAssignableFrom, maybe we can do something different for that call, if needed. isAssignableFrom, I believe is the Class equivilent of isInstance. leouser --- Leo User <leo...@ya...> wrote: > Another technique cuts things down further: > 500 > 449 > 429 > 428 > 430 > 432 > 435 > 444 > 429 > 429 > 438 > > This would be swapping out the assignable set check > with a simple class generation scheme for object of > type InstanceOf. The generated classes are very > simple, they just do this: > > return o instanceof YourClassHere > > Which is what the code is crying for. Ive seen this > take the timings down to 414. Of course, again, > this > raises the question of expense. How much more > memory > are we going to eat by having InstanceOfs floating > around. Thankfully, there will just be one per > class. > From looking at the core package there are other > possible candidate classes that could use this, so > maybe it would pay for itself beyond the PyInstance. > > profile leaders: > 15.3% 57 + 0 java.util.HashMap.get > 9.1% 34 + 0 > org.python.core.PyInstance.__findattr__ > 8.1% 25 + 5 > org.python.core.PyJavaMethodProxy.convert > 8.1% 30 + 0 > org.python.core.PyInstance.getInstanceOf > > leouser > > --- Leo User <leo...@ya...> wrote: > > > Some more improvements: > > 579 > > 497 > > 485 > > 484 > > 483 > > 485 > > 495 > > 486 > > 484 > > 485 > > 483 > > > > What's happening here is that in the > > PyJavaMethodProxy > > Ive concoted a specialised version of java2py. > This > > one instead of checking the argument to see what > it > > is > > already has an argument number assigned to it. > This > > feeds into a switch statement that converts it > like, > > Py.java2py does. The assigned number is created > > when > > the PyJavaMethodProxy subclass is synthesized. > This > > roughly cuts 50-60 more milliseconds from the > test. > > > > The new time eaters are: > > 13.4% 56 + 0 > > org.python.core.PyInstance.isInstance > > 11.0% 46 + 0 java.util.HashMap.get > > 9.8% 25 + 16 > > org.python.core.PyClass.lookupGivingClass > > 7.4% 31 + 0 > org.python.core.Py.java2py > > 6.9% 20 + 9 > > org.python.core.PyJavaMethodProxy.convert > > 5.3% 22 + 0 > > org.python.core.PyJavaMethodProxy.__call__ > > 5.3% 22 + 0 > > org.python.core.PyObject.__get__ > > 5.0% 21 + 0 > > org.python.core.PyJavaClass.lookupGivingClass > > > > > > Maybe these lookupGivingClass methods will be > > worthwhile exploring. > > > > leouser > > > > > > --- Leo User <leo...@ya...> wrote: > > > > > The fromClass call was being invoked each time a > > > special class in the PyJavaMethodProxy was being > > > instantiated. Cutting that out, stops that > method > > > from being invoked and reduces the times, again, > > to: > > > 641 > > > 561 > > > 551 > > > 556 > > > 550 > > > 549 > > > 558 > > > 539 > > > 541 > > > 543 > > > 546 > > > > > > > Hmm, it looks like some elementary > changes > > > to > > > > > the > > > > > > > caching system can speed things up even > > > > further. > > > > > > > > > > > > From > > > > > > > running my TestLoopTime.py test, Im > seeing > > a > > > > > drop > > > > > > of > > > > > > > about 300 more milliseconds from > > execution. > > > > > > > > > > > > > > UNMODIFIED TIMES: > > > > > > > 2087 > > > > > > > 2023 > > > > > > > 2015 > > > > > > > 2022 > > > > > > > 2017 > > > > > > > > > > > > > __________________________________________________ > > > Do You Yahoo!? > > > Tired of spam? Yahoo! Mail has the best spam > > > protection around > > > http://mail.yahoo.com > > > > > > > > > ------------------------------------------------------------------------- > > > Take Surveys. Earn Cash. Influence the Future of > > IT > > > Join SourceForge.net's Techsay panel and you'll > > get > > > the chance to share your > > > opinions on IT & business topics through brief > > > surveys - and earn cash > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > _______________________________________________ > > > Jython-dev mailing list > > > Jyt...@li... > > > > > > https://lists.sourceforge.net/lists/listinfo/jython-dev > > > > > > > > > __________________________________________________ > > Do You Yahoo!? > > Tired of spam? Yahoo! Mail has the best spam > > protection around > > http://mail.yahoo.com > > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of > IT > > Join SourceForge.net's Techsay panel and you'll > get > > the chance to share your > > opinions on IT & business topics through brief > > surveys - and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Jython-dev mailing list > > Jyt...@li... > > > https://lists.sourceforge.net/lists/listinfo/jython-dev > > > > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam > protection === message truncated === __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2007-01-03 19:44:50
|
Ditching HashMap lookup for InstanceOfs appears to pay further time dividends: 451 412 389 395 390 387 390 388 399 390 388 What is occuring is that I am supplementing the PyJavaMethodProxy instance with an InstanceOf for each Class stored in it. This allows us to pass the InstanceOf along with the class to __tojava__. Since its being passed in, there is no need to look it up in the map of InstanceOfs. If we hit the 200's, it will be a nifty thing to see, that will be approximately 1/10th of the time it takes to run the test on an unmodified jython. Getting there may be hard to do though, the big consumers are starting to be the very simple methods we have generated. If we make __findattr__ quicker, it can only pay us like 15-20 milliseconds of execution time. Though if we were able to get all the "find" methods to be quicker yet, it could payoff. From looking at the profile, it seems that 30%+ of the time is spent looking for things. leouser --- Leo User <leo...@ya...> wrote: > The list of other classes that could be helped by > InstaceOf are: > PyInstance > PyObject > PyString > PyArray > > each one of these uses the dreaded Class.isInstance > call. Im not sure if InstanceOf can be a drop in > replacement for isAssignableFrom, maybe we can do > something different for that call, if needed. > isAssignableFrom, I believe is the Class equivilent > of > isInstance. > > leouser > > > --- Leo User <leo...@ya...> wrote: > > > Another technique cuts things down further: > > 500 > > 449 > > 429 > > 428 > > 430 > > 432 > > 435 > > 444 > > 429 > > 429 > > 438 > > > > This would be swapping out the assignable set > check > > with a simple class generation scheme for object > of > > type InstanceOf. The generated classes are very > > simple, they just do this: > > > > return o instanceof YourClassHere > > > > Which is what the code is crying for. Ive seen > this > > take the timings down to 414. Of course, again, > > this > > raises the question of expense. How much more > > memory > > are we going to eat by having InstanceOfs floating > > around. Thankfully, there will just be one per > > class. > > From looking at the core package there are other > > possible candidate classes that could use this, so > > maybe it would pay for itself beyond the > PyInstance. > > > > > > > > Hmm, it looks like some elementary > > changes > > > > to > > > > > > the > > > > > > > > caching system can speed things up > even > > > > > further. > > > > > > > > > > > > > > From > > > > > > > > running my TestLoopTime.py test, Im > > seeing > > > a > > > > > > drop > > > > > > > of > > > > > > > > about 300 more milliseconds from > > > execution. > > > > > > > > > > > > > > > > UNMODIFIED TIMES: > > > > > > > > 2087 > > > > > > > > 2023 > > > > > > > > 2015 > > > > > > > > 2022 > > > > > > > > 2017 > > > > > > > > > > > > > > > > > > __________________________________________________ > > > > Do You Yahoo!? > > > > Tired of spam? Yahoo! Mail has the best spam > > > > protection around > > > > http://mail.yahoo.com > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > Take Surveys. Earn Cash. Influence the Future > of > > > IT > > > > Join SourceForge.net's Techsay panel and > you'll > > > get > > > > the chance to share your > > > > opinions on IT & business topics through brief > > > > surveys - and earn cash > > > > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > _______________________________________________ > > > > Jython-dev mailing list > > > > Jyt...@li... > > > > > === message truncated === __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2007-01-03 21:35:54
|
well, another techique shaves more time off: 311 254 240 239 239 239 242 239 241 242 242 this one is keeping a simple cache of the last attributes requested and returned. If the attribute is found in the cache, that is returned. Though it slices another 100 milliseconds off I have to wonder about this. It is going to require some more management on the cache to make it a viable technique. What if the attribute is deleted or altered? The cache may need to be invalidated in this case. This, I think, is doable at the PyInstance instance but becomes a little more problematic when the class changes. Ill have to think about this one, but I was not expecting that big of a jump. I stressed in another post elsewhere that lookups seemed to be taking 1/3rd of the execution time, this perception looks right at this moment. Geesh, maybe breaking into the 100's isn't that odd of an idea... leouser --- Leo User <leo...@ya...> wrote: > Ditching HashMap lookup for InstanceOfs appears to > pay > further time dividends: > 451 > 412 > 389 > 395 > 390 > 387 > 390 > 388 > 399 > 390 > 388 > > What is occuring is that I am supplementing the > PyJavaMethodProxy instance with an InstanceOf for > each > Class stored in it. This allows us to pass the > InstanceOf along with the class to __tojava__. > Since > its being passed in, there is no need to look it up > in > the map of InstanceOfs. If we hit the 200's, it > will > be a nifty thing to see, that will be approximately > 1/10th of the time it takes to run the test on an > unmodified jython. Getting there may be hard to do > though, the big consumers are starting to be the > very > simple methods we have generated. If we make > __findattr__ quicker, it can only pay us like 15-20 > milliseconds of execution time. Though if we were > able to get all the "find" methods to be quicker > yet, > it could payoff. From looking at the profile, it > seems that 30%+ of the time is spent looking for > things. > > leouser > > > --- Leo User <leo...@ya...> wrote: > > > The list of other classes that could be helped by > > InstaceOf are: > > PyInstance > > PyObject > > PyString > > PyArray > > > > each one of these uses the dreaded > Class.isInstance > > call. Im not sure if InstanceOf can be a drop in > > replacement for isAssignableFrom, maybe we can do > > something different for that call, if needed. > > isAssignableFrom, I believe is the Class > equivilent > > of > > isInstance. > > > > leouser > > > > > > --- Leo User <leo...@ya...> wrote: > > > > > Another technique cuts things down further: > > > 500 > > > 449 > > > 429 > > > 428 > > > 430 > > > 432 > > > 435 > > > 444 > > > 429 > > > 429 > > > 438 > > > > > > This would be swapping out the assignable set > > check > > > with a simple class generation scheme for object > > of > > > type InstanceOf. The generated classes are very > > > simple, they just do this: > > > > > > return o instanceof YourClassHere > > > > > > Which is what the code is crying for. Ive seen > > this > > > take the timings down to 414. Of course, again, > > > this > > > raises the question of expense. How much more > > > memory > > > are we going to eat by having InstanceOfs > floating > > > around. Thankfully, there will just be one per > > > class. > > > From looking at the core package there are > other > > > possible candidate classes that could use this, > so > > > maybe it would pay for itself beyond the > > PyInstance. > > > > > > > > > Hmm, it looks like some elementary > > > changes > > > > > to > > > > > > > the > > > > > > > > > caching system can speed things up > > even > > > > > > further. > > > > > > > > > > > > > > > > From > > > > > > > > > running my TestLoopTime.py test, Im > > > seeing > > > > a > > > > > > > drop > > > > > > > > of > > > > > > > > > about 300 more milliseconds from > > > > execution. > > > > > > > > > > > > > > > > > > UNMODIFIED TIMES: > > > > > > > > > 2087 > > > > > > > > > 2023 > > > > > > > > > 2015 > > > > > > > > > 2022 > > > > > > > > > 2017 > > > > > > > > > > > > > > > > > > > > > > > > __________________________________________________ > > > > > Do You Yahoo!? > > > > > Tired of spam? Yahoo! Mail has the best > spam > > > > > protection around > > > > > http://mail.yahoo.com > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > Take Surveys. Earn Cash. Influence the > Future > > of > > > > IT > > > > > Join SourceForge.net's Techsay panel and > > you'll > > > > get > > > > > the chance to share your > > > > > opinions on IT & business topics through > brief > > > > > surveys - and earn cash > > > > > > > > > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > > _______________________________________________ > > > > > Jython-dev mailing list > > > > > Jyt...@li... > > > > > > > > === message truncated === > > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam > protection around > http://mail.yahoo.com > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get > the chance to share your > opinions on IT & business topics through brief > surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Leo U. <leo...@ya...> - 2007-01-04 19:51:25
|
Hmm, its odd to see that from profiling that we are back at the start of the journey: 15.7% 31 + 3 org.python.core.PyJavaMethodProxy.to 12.4% 27 + 0 org.python.core.PyJavaMethodProxy.__call__ 7.4% 15 + 1 org.python.core.PyJavaMethodProxy$4.j2p "to" converts a PyObject array to java Objects. This one enables us to actually invoke the java method with java arguments. __call__ is one of your "__call__"s you see in PyObject. Its the basic entry to invoking. "j2p" is one of the java to python conversion methods. It takes a return value and returns it to the Jython system. Before "j2p", the code was using Py.java2py which is evil. "j2p"'s are simple methods that the code uses when it knows ahead of time the type of return value. When you profile an umodified jython against this particular test it isn't the reflected methods that appear to eat up most of the time, its lookup: 20.1% 269 + 1 org.python.core.PyClass.lookupGivingClass and System.identityHashCode: 22.6% 0 + 304 java.lang.System.identityHashCode Im not sure if there is anything else obvious to cut/improve. Its possible that supplementing the __tojava__ method in PyObject with precise conversion methods: __toString__, __toInteger__, etc could help but then that would raise the question of having another targeting scheme. Then again, reducing the calls to Py.java2Py to methods that do one thing appears to have assisted nicely so it may not be that bad of an idea to look at. Given that the test is running at 1/10th of the time it takes to run on the unmodified, consolidating the wins and making sure that they work beyond the confines of the simple exercise loop here may be in order. leouser --- Leo User <leo...@ya...> wrote: > > > > > > > > > > > > > > > > > > > > UNMODIFIED TIMES: > > > > > > > > > > 2087 > > > > > > > > > > 2023 > > > > > > > > > > 2015 > > > > > > > > > > 2022 > > > > > > > > > > 2017 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > __________________________________________________ > > > > > > Do You Yahoo!? > > > > > > Tired of spam? Yahoo! Mail has the best > > spam > > > > > > protection around > > > > > > http://mail.yahoo.com > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > > Take Surveys. Earn Cash. Influence the > > Future > > > of > === message truncated === __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
From: Jeff N. <jn...@ac...> - 2007-01-05 06:18:38
|
I had been following this thread with interest before Christmas as we are having performance problems with Jython, but now (after I've had too much figgy pudding) I've forgotten exactly what you were speeding up. Is this just initialization or does it apply to every method dispatch? Thanks, Jeff. =20 > -----Original Message----- > From: jyt...@li...=20 > [mailto:jyt...@li...] On Behalf=20 > Of Leo User > Sent: Thursday, January 04, 2007 11:51 AM > To: jyt...@li... > Subject: Re: [Jython-dev] Some ok progress with the=20 > PyJavaMethodProxyexperiment >=20 > Hmm, >=20 > its odd to see that from profiling that we are back at the=20 > start of the journey: > 15.7% 31 + 3 =20 > org.python.core.PyJavaMethodProxy.to > 12.4% 27 + 0 =20 > org.python.core.PyJavaMethodProxy.__call__ > 7.4% 15 + 1 =20 > org.python.core.PyJavaMethodProxy$4.j2p >=20 > "to" converts a PyObject array to java Objects. This one=20 > enables us to actually invoke the java method with java arguments. >=20 > __call__ is one of your "__call__"s you see in PyObject. Its=20 > the basic entry to invoking. >=20 > "j2p" is one of the java to python conversion methods. > It takes a return value and returns it to the Jython system.=20 > Before "j2p", the code was using Py.java2py which is evil.=20 > "j2p"'s are simple methods that the code uses when it knows=20 > ahead of time the type of return value. >=20 > When you profile an umodified jython against this particular=20 > test it isn't the reflected methods that appear to eat up=20 > most of the time, its lookup: > 20.1% 269 + 1 =20 > org.python.core.PyClass.lookupGivingClass > and System.identityHashCode: > 22.6% 0 + 304 =20 > java.lang.System.identityHashCode >=20 >=20 > Im not sure if there is anything else obvious to cut/improve.=20 > Its possible that supplementing the __tojava__ method in=20 > PyObject with precise conversion > methods: > __toString__, __toInteger__, etc could help but then that=20 > would raise the question of having another targeting scheme. =20 > Then again, reducing the calls to Py.java2Py to methods that=20 > do one thing appears to have assisted nicely so it may not be=20 > that bad of an idea to look at. Given that the test is=20 > running at 1/10th of the time it takes to run on the=20 > unmodified, consolidating the wins and making sure that they=20 > work beyond the confines of the simple exercise loop here may=20 > be in order. >=20 > leouser >=20 >=20 >=20 > --- Leo User <leo...@ya...> wrote: >=20 >=20 > > > > > > > > > > >=20 > > > > > > > > > > > UNMODIFIED TIMES: > > > > > > > > > > > 2087 > > > > > > > > > > > 2023 > > > > > > > > > > > 2015 > > > > > > > > > > > 2022 > > > > > > > > > > > 2017 > > > > > > >=20 > > > > > > >=20 > > > > > > >=20 > > > > > > > > > > > > > > > __________________________________________________ > > > > > > > Do You Yahoo!? > > > > > > > Tired of spam? Yahoo! Mail has the best > > > spam > > > > > > > protection around > > > > > > > http://mail.yahoo.com > > > > > > >=20 > > > > > > > > > > > > > > > > > > > > > > > > > > > > -------------------------------------------------------------- > ----------- > > > > > > > Take Surveys. Earn Cash. Influence the > > > Future > > > > of > >=20 > =3D=3D=3D message truncated =3D=3D=3D >=20 >=20 > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam protection=20 > around http://mail.yahoo.com=20 >=20 > -------------------------------------------------------------- > ----------- > Take Surveys. Earn Cash. Influence the Future of IT Join=20 > SourceForge.net's Techsay panel and you'll get the chance to=20 > share your opinions on IT & business topics through brief=20 > surveys - and earn cash=20 > http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge &CID=3DDEVDEV > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev >=20 |
From: Leo U. <leo...@ya...> - 2007-01-05 14:51:18
|
This is the current test Ive been using as a simple way to see how the overhead of Java method invocation can be cut: import java.lang as jl st = jl.System.currentTimeMillis def zoo(): s = jl.String("MOOOOOOOOOOOO") x = st() for z in xrange(300000): s.toString() s.charAt(0) s.toString() s.charAt(0) x2 = st() print (x2 - x) return z for z in xrange(10): zoo() x = st() for z in xrange(300000): pass x2 = st() print (x2 - x) With an umodified version of Jython it takes about 2 seconds per invocation of "zoo" to execute. With the PyJavaMethodProxy version its now taking around 210-230 milliseconds to execute. From looking at the last loop it appears that it roughly adds 100 Milliseconds to execution time. leouser --- Jeff Norton <jn...@ac...> wrote: > I had been following this thread with interest > before Christmas as we > are having > performance problems with Jython, but now (after > I've had too much figgy > pudding) > I've forgotten exactly what you were speeding up. > Is this just > initialization or > does it apply to every method dispatch? > > Thanks, > Jeff. > > > > -----Original Message----- > > From: jyt...@li... > > [mailto:jyt...@li...] > On Behalf > > Of Leo User > > Sent: Thursday, January 04, 2007 11:51 AM > > To: jyt...@li... > > Subject: Re: [Jython-dev] Some ok progress with > the > > PyJavaMethodProxyexperiment > > > > Hmm, > > > > its odd to see that from profiling that we are > back at the > > start of the journey: > > 15.7% 31 + 3 > > org.python.core.PyJavaMethodProxy.to > > 12.4% 27 + 0 > > org.python.core.PyJavaMethodProxy.__call__ > > 7.4% 15 + 1 > > org.python.core.PyJavaMethodProxy$4.j2p > > > > "to" converts a PyObject array to java Objects. > This one > > enables us to actually invoke the java method with > java arguments. > > > > __call__ is one of your "__call__"s you see in > PyObject. Its > > the basic entry to invoking. > > > > "j2p" is one of the java to python conversion > methods. > > It takes a return value and returns it to the > Jython system. > > Before "j2p", the code was using Py.java2py which > is evil. > > "j2p"'s are simple methods that the code uses when > it knows > > ahead of time the type of return value. > > > > When you profile an umodified jython against this > particular > > test it isn't the reflected methods that appear to > eat up > > most of the time, its lookup: > > 20.1% 269 + 1 > > org.python.core.PyClass.lookupGivingClass > > and System.identityHashCode: > > 22.6% 0 + 304 > > java.lang.System.identityHashCode > > > > > > Im not sure if there is anything else obvious to > cut/improve. > > Its possible that supplementing the __tojava__ > method in > > PyObject with precise conversion > > methods: > > __toString__, __toInteger__, etc could help but > then that > > would raise the question of having another > targeting scheme. > > Then again, reducing the calls to Py.java2Py to > methods that > > do one thing appears to have assisted nicely so it > may not be > > that bad of an idea to look at. Given that the > test is > > running at 1/10th of the time it takes to run on > the > > unmodified, consolidating the wins and making sure > that they > > work beyond the confines of the simple exercise > loop here may > > be in order. > > > > leouser > > > > > > > > --- Leo User <leo...@ya...> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > UNMODIFIED TIMES: > > > > > > > > > > > > 2087 > > > > > > > > > > > > 2023 > > > > > > > > > > > > 2015 > > > > > > > > > > > > 2022 > > > > > > > > > > > > 2017 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > __________________________________________________ > > > > > > > > Do You Yahoo!? > > > > > > > > Tired of spam? Yahoo! Mail has the > best > > > > spam > > > > > > > > protection around > > > > > > > > http://mail.yahoo.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -------------------------------------------------------------- > > ----------- > > > > > > > > Take Surveys. Earn Cash. Influence the > > > > Future > > > > > of > > > > > === message truncated === > > > > > > __________________________________________________ > > Do You Yahoo!? > > Tired of spam? Yahoo! Mail has the best spam > protection > > around http://mail.yahoo.com > > > > > -------------------------------------------------------------- > > ----------- > > Take Surveys. Earn Cash. Influence the Future of > IT Join > > SourceForge.net's Techsay panel and you'll get the > chance to > > share your opinions on IT & business topics > through brief > > surveys - and earn cash > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge > &CID=DEVDEV > > _______________________________________________ > > Jython-dev mailing list > > Jyt...@li... > > > https://lists.sourceforge.net/lists/listinfo/jython-dev > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get > the chance to share your > opinions on IT & business topics through brief > surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |