Thread: [Pydev-users] [Users] out of memory error
Brought to you by:
fabioz
Menu
▾
▴
pydev-users
|
From: SourceForge.net <no...@so...> - 2011-10-05 00:27:51
|
The following forum message was posted by rekveld at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: Hi, since about a day I have recurrent 'Out of memory errors' when using PyDev. All is fine when I start, but I can see the CPU-activity of the java process gradually going up until Eclipse slowly freezes and error alerts appear. I'm not aware I have changed anything, there might have been a java-update I missed, but I've tried it with two different vm's and they both have the same problem (the Sun jvm is faster, so it takes less time for eclipse to freeze). I tried installing Indigo with the latest PyDev downloaded 10 minutes ago, but the problem persists. I tried commenting out large imports, gave more memory, removing and re-adding the interpreter, but none of these have made a difference. help ? any pointers welcome, Joost. Am I the only one experiencing this at the moment ? |
|
From: SourceForge.net <no...@so...> - 2011-10-05 00:29:30
|
The following forum message was posted by rekveld at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: forgot to mention i'm on Ubuntu 11.04.. |
|
From: SourceForge.net <no...@so...> - 2011-10-05 01:46:12
|
The following forum message was posted by fabioz at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: How much memory are you giving it before it runs out of memory? See: http://stackoverflow.com/questions/142357/what-are-the-best-jvm-settings-for-eclipse for a reference configuring vm settings. If the problem persists, please provide a memory dump of your use-case (see: http://wiki.eclipse.org/index.php/MemoryAnalyzer for details on getting a memory dump -- zip it, upload it somewhere and pass me the link). Cheers, Fabio |
|
From: SourceForge.net <no...@so...> - 2011-10-05 07:11:03
|
The following forum message was posted by rekveld at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: Hi Fabio, in the eclipse.ini I have the following parameters: --launcher.XXMaxPermSize 512m --launcher.defaultAction openFile -vmargs -Xms128m -Xmx1024m which should be enough, no ? And what mystifies me the most is that these problems started completely out-of-the-blue; i've been using PyDev/Eclipse for years and it's great ! I am going to look into this MemoryAnalyzer thing now; at first sight the documentation of it seems a bit lacking for my kind of user.. J. |
|
From: SourceForge.net <no...@so...> - 2011-10-05 10:41:30
|
The following forum message was posted by rekveld at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: Hi Fabio, finally managed to do a 554MB heapdump and uploaded it to http://rekveld.home.xs4all.nl/stuff (i have to run now and it still has 30mins of uploading to go) I modified the settings a bit in order to avoid having to upload a 1.2GB dump, but I don't think that makes a difference for tracking down the problem. And for those who ever need to do this, what I figured out is to add these two lines to your eclipse.ini file. The last one specifies the directory and it is wise to make a directory especially for it, since the analysis produces lots of little files you don't want to pollute your working directory or home folder. It took me a while to find this out, since all the documentation is mostly explaining you how to find memoryleaks in your own code, not in the IDE you're using. I had a look and it seems to have to do with code analysis. A couple of times Eclipse froze on completing an Open_GL attribute, and in the system monitor I could see it filling up 1.2G of memory in 2 or 3 minutes. --launcher.XXMaxPermSize 256m openFile -vmargs -Xms128m -Xmx384m -Xss4m -XX:PermSize=128m -XX:MaxPermSize=384m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/your/path/here |
|
From: SourceForge.net <no...@so...> - 2011-10-05 13:09:21
|
The following forum message was posted by fabioz at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: From taking a look at the heap it really looks that PyDev got lost when searching things in pyopengl (it recreates many times a module searching for it until you run out of memory... if you had even more memory allocated, it would probably end up raising a stack overflow error), so, it seems that some specific construct triggered that (will have to analyze it better to know what could trigger that). Which pyopengl version are you using? I'm not sure if your project is closed or open source, but if it's open source, it'd help if you could upload the project that has this problem + the python interpreter with all the libs you're using (if not it's open source, having only the interpreter+libs could be helpful too -- if you can't do that either, can you specify the python version, pyopengl version you're using)? Cheers, Fabio |
|
From: SourceForge.net <no...@so...> - 2011-10-05 14:06:14
|
The following forum message was posted by rekveld at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: Ha Fabio, as usual, thanks for the speedy and clarifying response. For now, the project is closed source. I am an artist and I am mostly coding things for my own use, but there are large parts of this project of which I hope they can be of general use and which I waould like to share. But deadlines first :) From your pointer I think I will be able to piece together a minimum case that would trigger this too. I am away from my box now, but I will try what I have to do to trigger it starting from a new, empty workspace. I recently started using pygtkglext, from what you write I get the feeling the problem might be related to that. More detailed info soon ! many thanks, Joost. |
|
From: SourceForge.net <no...@so...> - 2011-10-05 22:26:37
|
The following forum message was posted by rekveld at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: Hi Fabio, I've been playing around a bit in a new workspace and I found my error is relatively simple to reproduce after your previous message: this file: http://python-gtkglext1.sourcearchive.com/documentation/1.1.0-4/coolwave_8py-sou rce.html is one of the examples that comes with PyGTKGLExt. If I start adding lines like glEnable(GL_WORLDPEACE) I can see in the systemmonitor that the java thread starts going up to 100% and memory usage starts increasing rapidly. (Strange is that just now I tried again after giving Eclipse a lot less memory (in order to give you a smaller heapdump) and then it also happens, but it takes muuuuch longer to reach the Out of Memory point, and sometimes I don't have the patience to get there. That is not what I expected, but not a solution I guess, it does slow everything down.) I am on Python 2.6.6 using pygtkglext-1.1.0, pyOpenGL 3.0.1, pyGTK 2.0, what else would you need to know ? What I just discovered is that in the import section it says [code]from gtk.gtkgl.apputils import * from OpenGL.GL import *[/code] and gtk.gtkgl.apputils imports/redefines (I don't know which, I guess the latter) A LOT of the definitions in OpenGL.GL I suspect that might have to do with the problem ? It is confusing to use wildcard imports like in this example; in my own project I import all the OpenGL stuff I need from gtk.gtkgl.apputils, and only two functions from OpenGL.GLU, nothing at all from OpenGL.GL. (I even tried importing all the OpenGL definitions one by one, without a wildcard, but that did not change anything). I hope this gives you enough info to reproduce ? Or should I try excluding libs from my interpreter ? many thanks, Joost. |
|
From: SourceForge.net <no...@so...> - 2011-10-23 23:23:04
|
The following forum message was posted by fabioz at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: Just to note: I tried reproducing in windows and wasn't able to get to the OutOfMemory condition... I may have to setup a linux install (maybe something is different there). |
|
From: SourceForge.net <no...@so...> - 2011-10-24 00:34:39
|
The following forum message was posted by fabioz at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: Ok, just got setup linux, gotten gtk/gtkgl/pyopengl, imported the example you passed ( http://python-gtkglext1.sourcearchive.com/documentation/1.1.0-4/coolwav e_8py-source.html ) up to the point to run it and tried to work in the file and (unfortunately) all went well (no Out of memory -- in fact, the heap reports as being around 90 MB). I'll try to think of something else to try to reproduce it... meanwhile, can you pass me some more info? go to window > preferences > pydev > logging and mark 'enable logging for code-completion', then, try to do a code-completion in the case that'd give you an out of memory error and pass me the output you get in the console in that case. Cheers, Fabio |
|
From: SourceForge.net <no...@so...> - 2011-12-09 20:14:50
|
The following forum message was posted by qu9542 at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: I got the exact same problem with pyOpenGL and PyDev in Eclipse. Ubuntu 10.04 64bit Eclipse Indigo + PyDev When I try to open "pyOpenGL" example py file @ http://pypi.python.org/pypi/PyOpenGL-Demo. Eclipse IDE hangs, and java is using 1.2GB of my memory, 100% CPU in 1~2 minutes. Then I got "gc overhead limit exceeded" and "out of memory" error. here is my eclipse.ini file: -------------------------- -startup plugins/org.eclipse.equinox.launcher_1.2.0.v20110502.jar --launcher.library plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.100.v20110505 -showsplash org.eclipse.platform --launcher.XXMaxPermSize 512m --launcher.defaultAction openFile -vmargs -XX:MaxPermSize=512m -Xms512m -Xmx1024m -------------------------- Q |
|
From: SourceForge.net <no...@so...> - 2011-12-10 20:16:52
|
The following forum message was posted by qu9542 at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: [update] Here are the steps to reproduce this problem [1] Launch Eclipse to a new workspace [2] Create a new PyDev project (add /usr/bin/python as python interpreter) [3] download pygtkglext-1.1.0.tar.gz @ https://sourceforge.net/projects/gtkglext/files/pygtkglext/1.1.0/pygtkglext-1. 1.0.tar.gz/download [4] unzip the file, and Copy and paste the example folder into the new PyDev project folder. [5] Refresh PyDev project, and open one .py file (e.g., "NeHe6Multi.py") in the Eclipse Editor. [6] wait... check the memory usage by java... growing ...growing... very quickly... When memory by java reaches 1.2GB, eclipse hangs... Even you can quit eclipse, this java process will stay as zombie process. Q |
|
From: SourceForge.net <no...@so...> - 2011-12-10 20:30:34
|
The following forum message was posted by qu9542 at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: [Update] If no .py file is displayed in the Eclipse Editor, then everything is OK. I can right click on the .py file, and choose "run as" "python run", program will run. As long as you open .py file in the Eclipse editor, Eclipse will hang. |
|
From: SourceForge.net <no...@so...> - 2011-12-10 21:59:43
|
The following forum message was posted by qu9542 at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: [Update] Here is the log file for Eclipse (found in the project .metadata folder): !ENTRY org.python.pydev.core 4 4 2011-12-09 13:51:27.066 !MESSAGE Java heap space !STACK 0 java.lang.OutOfMemoryError: Java heap space at org.python.pydev.parser.grammarcommon.AbstractTreeBuilder.openNode(AbstractT reeBuilder.java:187) at org.python.pydev.parser.grammar27.PythonGrammar27.comparison(PythonGrammar27 .java:4089) at org.python.pydev.parser.grammar27.PythonGrammar27.not_test(PythonGrammar27.j ava:4078) at org.python.pydev.parser.grammar27.PythonGrammar27.and_test(PythonGrammar27.j ava:3984) at org.python.pydev.parser.grammar27.PythonGrammar27.or_test(PythonGrammar27.ja va:3940) at org.python.pydev.parser.grammar27.PythonGrammar27.test(PythonGrammar27.java: 3859) at org.python.pydev.parser.grammar27.PythonGrammar27.listmaker(PythonGrammar27. java:5969) at org.python.pydev.parser.grammar27.PythonGrammar27.atom(PythonGrammar27.java: 5173) at org.python.pydev.parser.grammar27.PythonGrammar27.atomtrailer(PythonGrammar2 7.java:4909) at org.python.pydev.parser.grammar27.PythonGrammar27.power(PythonGrammar27.java :4868) at org.python.pydev.parser.grammar27.PythonGrammar27.factor(PythonGrammar27.jav a:4855) at org.python.pydev.parser.grammar27.PythonGrammar27.term(PythonGrammar27.java: 4603) at org.python.pydev.parser.grammar27.PythonGrammar27.arith_expr(PythonGrammar27 .java:4522) at org.python.pydev.parser.grammar27.PythonGrammar27.shift_expr(PythonGrammar27 .java:4441) at org.python.pydev.parser.grammar27.PythonGrammar27.and_expr(PythonGrammar27.j ava:4398) at org.python.pydev.parser.grammar27.PythonGrammar27.xor_expr(PythonGrammar27.j ava:4355) at org.python.pydev.parser.grammar27.PythonGrammar27.expr(PythonGrammar27.java: 4312) at org.python.pydev.parser.grammar27.PythonGrammar27.comparison(PythonGrammar27 .java:4094) at org.python.pydev.parser.grammar27.PythonGrammar27.not_test(PythonGrammar27.j ava:4078) at org.python.pydev.parser.grammar27.PythonGrammar27.and_test(PythonGrammar27.j ava:3984) at org.python.pydev.parser.grammar27.PythonGrammar27.or_test(PythonGrammar27.ja va:3940) at org.python.pydev.parser.grammar27.PythonGrammar27.test(PythonGrammar27.java: 3859) at org.python.pydev.parser.grammar27.PythonGrammar27.SmartTestList(PythonGramma r27.java:5773) at org.python.pydev.parser.grammar27.PythonGrammar27.expr_stmt(PythonGrammar27. java:1738) at org.python.pydev.parser.grammar27.PythonGrammar27.small_stmt(PythonGrammar27 .java:838) at org.python.pydev.parser.grammar27.PythonGrammar27.simple_stmt(PythonGrammar2 7.java:778) at org.python.pydev.parser.grammar27.PythonGrammar27.stmt(PythonGrammar27.java: 753) at org.python.pydev.parser.grammar27.PythonGrammar27.file_input(PythonGrammar27 .java:201) at org.python.pydev.parser.grammar27.PythonGrammar27.file_input(PythonGrammar27 .java:1) at org.python.pydev.parser.PyParser.reparseDocument(PyParser.java:647) at org.python.pydev.editor.codecompletion.revisited.modules.AbstractModule.crea teModuleFromDoc(AbstractModule.java:267) at org.python.pydev.editor.codecompletion.revisited.modules.AbstractModule.crea teModule(AbstractModule.java:244) |
|
From: SourceForge.net <no...@so...> - 2011-12-14 14:30:21
|
The following forum message was posted by qu9542 at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: [workaround] Eclipse->Windows->Preference->PyDev->Interpreter - Python->Forced Builtins, remove "OpenGL" from the list. |
|
From: SourceForge.net <no...@so...> - 2011-12-14 16:02:55
|
The following forum message was posted by fabioz at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: Ok, I think I've been able to fix this issue. Please get the nightly build and see if it works (the current nightly should be the final build -- see: http://pydev.blogspot.com/2011/12/preparing-for-pydev-230.html ) Cheers, Fabio |
|
From: SourceForge.net <no...@so...> - 2011-12-17 02:27:39
|
The following forum message was posted by qu9542 at http://sourceforge.net/projects/pydev/forums/forum/293649/topic/4739830: Hi Fabio, That's great. I tried the new upgrade of PyDev 2.3 for about 1 hour (add "OpenGL" back to Forced Builtins), everything is OK. Problem is solved! Thanks a lot for your excellent work!! best regards & have a good weekend! Q |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X