Thread: [PyOpenGL-Users] crash on import on Mac OS X after xml-generation merge explained
Brought to you by:
mcfletch
Menu
▾
▴
pyopengl-users
|
From: rndblnch <rnd...@gm...> - 2014-01-21 11:48:00
|
hello,
i finally managed to get some time to look at the early crash on Mac OS X
that occurs on "from OpenGL import GL" since the xml-generation merge.
it turns out that somehow, the import process triggers a call to glGetString.
and if no opengl context is initialized at this point, it segfaults.
however, if the import is done after a proper context is initialized, the
import works and seams functional (no extensive testing though).
the minimal program below works for me (notice the gl import after the
glutCreateWindow).
hope this helps,
renaud
from OpenGL.GLUT import *
glutInit()
glutCreateWindow("test")
from OpenGL.GL import *
def display():
glClear(GL_COLOR_BUFFER_BIT)
glutSwapBuffers()
def reshape(w, h):
glViewport(0, 0, w, h)
glutPostRedisplay()
glutDisplayFunc(display)
glutReshapeFunc(reshape)
glutMainLoop()
|
|
From: Mike C. F. <mcf...@vr...> - 2014-01-21 15:33:48
|
On 01/21/2014 06:47 AM, rndblnch wrote: > hello, > > i finally managed to get some time to look at the early crash on Mac OS X > that occurs on "from OpenGL import GL" since the xml-generation merge. > it turns out that somehow, the import process triggers a call to glGetString. > and if no opengl context is initialized at this point, it segfaults. > however, if the import is done after a proper context is initialized, the > import works and seams functional (no extensive testing though). > the minimal program below works for me (notice the gl import after the > glutCreateWindow). Thanks Renaud, That gives me an idea how to track it down. The issue will likely be in the extension handling code, which was modified to support all of the new extension types in GLE, EGL, WGL, etc. Apparently I wound up moving a call to glGetString before context init. Appreciated, Mike -- ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com |
|
From: rndblnch <rnd...@gm...> - 2014-01-22 08:56:11
|
Mike C. Fletcher <mcfletch <at> vrplumber.com> writes: > > On 01/21/2014 06:47 AM, rndblnch wrote: > > hello, > > > > i finally managed to get some time to look at the early crash on Mac OS X > > that occurs on "from OpenGL import GL" since the xml-generation merge. > > it turns out that somehow, the import process triggers a call to glGetString. > > and if no opengl context is initialized at this point, it segfaults. > > however, if the import is done after a proper context is initialized, the > > import works and seams functional (no extensive testing though). > > the minimal program below works for me (notice the gl import after the > > glutCreateWindow). > Thanks Renaud, > > That gives me an idea how to track it down. The issue will likely be in > the extension handling code, which was modified to support all of the > new extension types in GLE, EGL, WGL, etc. Apparently I wound up moving > a call to glGetString before context init. yes, that was my intuition too. it is confirmed: if you comment out the 2 calls to glGetString in OpenGL/extension.py, OpenGL.GL loads (but no extension is available). however, i could not yet track the code path from import to the call to glGetString. i will try to look at it. renaud > > Appreciated, > Mike > |
|
From: Mike C. F. <mcf...@vr...> - 2014-01-24 20:13:29
|
On 01/21/2014 06:47 AM, rndblnch wrote: > hello, > > i finally managed to get some time to look at the early crash on Mac OS X > that occurs on "from OpenGL import GL" since the xml-generation merge. > it turns out that somehow, the import process triggers a call to glGetString. > and if no opengl context is initialized at this point, it segfaults. > however, if the import is done after a proper context is initialized, the > import works and seams functional (no extensive testing though). > the minimal program below works for me (notice the gl import after the > glutCreateWindow). > > hope this helps, Indeed it did. I tracked down the glGetString() calls, they happened because the image-wrappers do a bool( function ) on various functions which are GL_VERSION_1_2 *on import*, that is, they look at the functions and say "do you exist" and then decide whether to wrap them with convenience code. If you can test out bzr head on your machine it should confirm the fix. I'll start running through the Linux and Windows testing in the meantime to see if there's any regressions caused by it. Thanks, Mike -- ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com |
|
From: rndblnch <rnd...@gm...> - 2014-01-24 21:15:17
|
Mike C. Fletcher <mcfletch <at> vrplumber.com> writes: > Indeed it did. I tracked down the glGetString() calls, they happened > because the image-wrappers do a bool( function ) on various functions > which are GL_VERSION_1_2 *on import*, that is, they look at the > functions and say "do you exist" and then decide whether to wrap them > with convenience code. > > If you can test out bzr head on your machine it should confirm the fix. > I'll start running through the Linux and Windows testing in the meantime > to see if there's any regressions caused by it. it does not crash any more. however on my personal test suite (the same program converted step by step from using OpenGL 1. to modern OpenGL, see https://bitbucket.org/rndblnch/opengl-programmable/) it works until the introduction of vertex attributes (steps 01-direct.py to 06-perpixel.py work, but 07-attrib.py, 08-pbo.py, etc. give a black screen). i suspect that the pointer/offset parameter of the glXxxxPointer functions is not handled properly, i will look into it (i use ctypes arrays, no numpy). > Thanks, you are more than welcome, thank you for your work on PyOpenGL renaud > Mike |
|
From: Mike C. F. <mcf...@vr...> - 2014-01-28 15:33:04
|
On 14-01-24 04:14 PM, rndblnch wrote: > Mike C. Fletcher <mcfletch <at> vrplumber.com> writes: ... > it does not crash any more. > however on my personal test suite (the same program converted step by step > from using OpenGL 1. to modern OpenGL, see > https://bitbucket.org/rndblnch/opengl-programmable/) it works until the > introduction of vertex attributes (steps 01-direct.py to 06-perpixel.py > work, but 07-attrib.py, 08-pbo.py, etc. give a black screen). > i suspect that the pointer/offset parameter of the glXxxxPointer functions > is not handled properly, i will look into it (i use ctypes arrays, no numpy). I looked into it, and there was, indeed, a problem with the ctypes pointer handling. Basically we were getting pointer-to-pointer (i.e. the address of the pointer variable) instead of the pointer's value when doing a dataPointer() call on them. I've fixed that on bzr head, and all of your samples now seem to run correctly on bzr head. What's weird is that code hadn't changed; it seems that something higher-level meant that the code-path just wasn't being executed in the old PyOpenGL releases. By the way, do you have a tutorial (text) somewhere connected with your samples? And would you like the code linked into the PyOpenGL documentation samples (i.e. when someone looks at the glXxxxPointer functions they'd see a reference to your hg repository?) I fixed an (unrelated) bug that was preventing the various configuration flags (e.g. FULL_LOGGING) from running, and regenerated the OpenGL_accelerate wrappers with the new Cython release. I *think* we're fairly close to a beta-1 release for PyOpenGL 3.1 at this point. I still need to get some time to run the Windows tests and re-run the Linux tests with the updated Cython wrappers. Thanks, Mike -- ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com |
|
From: rndblnch <rnd...@gm...> - 2014-01-29 14:16:56
|
Mike C. Fletcher <mcfletch <at> vrplumber.com> writes: > I looked into it, and there was, indeed, a problem with the ctypes > pointer handling. Basically we were getting pointer-to-pointer (i.e. > the address of the pointer variable) instead of the pointer's value when > doing a dataPointer() call on them. I've fixed that on bzr head, and > all of your samples now seem to run correctly on bzr head. What's weird > is that code hadn't changed; it seems that something higher-level meant > that the code-path just wasn't being executed in the old PyOpenGL releases. ok, indeed, it works for me at rev 656, thanks. > By the way, do you have a tutorial (text) somewhere connected with your > samples? no, unfortunately not. this is support material for an opengl class, but the documents are in french :( > And would you like the code linked into the PyOpenGL > documentation samples (i.e. when someone looks at the glXxxxPointer > functions they'd see a reference to your hg repository?) sure, i would be honored :) two quick notes: - there are two branches, python2 is python2.5+ compatible, and default is python2.6+ and python3+ compatible with a single code base - demo 10-gl3.2core.py is MacOSX specific as it uses the GLUT_3_2_CORE_PROFILE capability. i guess that freeglut also offers the possibility to open a 3.2 core compatible context, i will look into this to provide a platform independent version of this code. > I fixed an (unrelated) bug that was preventing the various configuration > flags (e.g. FULL_LOGGING) from running, and regenerated the > OpenGL_accelerate wrappers with the new Cython release. > > I *think* we're fairly close to a beta-1 release for PyOpenGL 3.1 at > this point. I still need to get some time to run the Windows tests and > re-run the Linux tests with the updated Cython wrappers. i have done some testing with python3 too, and got a couple of syntax errors because the keyword "in" is used as argument name by two functions. the patch below shows where it happens (the fix should probably done at the generator level though): === modified file 'OpenGL/raw/GL/EXT/vertex_shader.py' --- OpenGL/raw/GL/EXT/vertex_shader.py 2013-11-22 15:15:02 +0000 +++ OpenGL/raw/GL/EXT/vertex_shader.py 2014-01-29 10:48:57 +0000 @@ -217,7 +217,7 @@ def glShaderOp3EXT(op,res,arg1,arg2,arg3):pass @_f @_p.types(None,_cs.GLuint,_cs.GLuint,_cs.GLenum,_cs.GLenum,_cs.GLenum,_cs.GLenum) -def glSwizzleEXT(res,in,outX,outY,outZ,outW):pass +def glSwizzleEXT(res,_in,outX,outY,outZ,outW):pass @_f @_p.types(None,_cs.GLuint,_cs.GLenum,_cs.GLuint,ctypes.c_void_p) def glVariantPointerEXT(id,type,stride,addr):pass @@ -248,4 +248,4 @@ def glVariantusvEXT(id,addr):pass @_f @_p.types(None,_cs.GLuint,_cs.GLuint,_cs.GLenum,_cs.GLenum,_cs.GLenum,_cs.GLenum) -def glWriteMaskEXT(res,in,outX,outY,outZ,outW):pass +def glWriteMaskEXT(res,_in,outX,outY,outZ,outW):pass and a quick remark: a Logger instance named log is imported into the global namespace, so it shadows the math function when doing something like that: """ from math import log from OpenGL.GLUT import * """ perhaps it could be possible to exclude it from * import (by setting a __all__, or using _log as its name)? not very important, i know... thanks again, i will be happy to do more testing once 3.1b1 is out (pypy and pypy3:) renaud > > Thanks, > Mike > |
|
From: Mike C. F. <mcf...@vr...> - 2014-01-30 21:09:49
|
On 14-01-29 09:16 AM, rndblnch wrote: ... > > i have done some testing with python3 too, and got a couple of syntax errors > because the keyword "in" is used as argument name by two functions. > the patch below shows where it happens (the fix should probably done at the > generator level though): I've fixed that, and run all of the parameter names through a test to filter out *any* python keyword (on the python (2.7) on which the generator is run). Note that I went with the suffixed rather than prefixed for (in_ rather than _in) for the names. Those were the only cases found with that check, btw. > and a quick remark: a Logger instance named log is imported into the global > namespace, so it shadows the math function when doing something like that: > """ > from math import log > from OpenGL.GLUT import * > """ > perhaps it could be possible to exclude it from * import (by setting a > __all__, or using _log as its name)? > not very important, i know... I've eliminated a couple of other name-pollution issues, but there's still a dozen or so "shouldn't be there" names in the final import namespace for GL. Unfortunately, using __all__ rather bloats the modules (there's a *lot* of names in the namespaces). So I've been doing that cleanup one-by-one with renames to _ prefixed names. I'll need to re-run the Linux tests, as those changes have modified *hundreds* of files, and then I still need to get the basic Win32 tests done to do the beta 1 release. Thanks, Mike -- ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com |
|
From: rndblnch <rnd...@gm...> - 2014-01-31 10:03:40
|
Mike C. Fletcher <mcfletch <at> vrplumber.com> writes: > I've eliminated a couple of other name-pollution issues, but there's > still a dozen or so "shouldn't be there" names in the final import > namespace for GL. Unfortunately, using __all__ rather bloats the > modules (there's a *lot* of names in the namespaces). So I've been doing > that cleanup one-by-one with renames to _ prefixed names. > > I'll need to re-run the Linux tests, as those changes have modified > *hundreds* of files, and then I still need to get the basic Win32 tests > done to do the beta 1 release. great, thanks! on my side, everything is ok on MacOSX 10.9.1, python2.7 & python3.3. the only regression i found now is when using pypy-2.1 or pypy3-2.1, but they are related to pypy's implementation of ctypes, and i am not sure that the pypy devs have any interest in fixing it as they are strongly supporting cffi now. i will report my finding to them anyway. by the way, pypy has never sped any of my opengl code... one example is the ctypes.c_char_p wrapper used in raw.GLUT: Python 2.7.3 (480845e6b1dd, Jul 31 2013, 10:58:28) [PyPy 2.1.0 with GCC 4.2.1 Compatible Clang Compiler] on darwin Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``random bugs in random (not a funny joke)'' >>>> from ctypes import c_char_p >>>> class STRING( c_char_p ): .... @classmethod .... def from_param( cls, value ): .... if isinstance( value, unicode ): .... value = value.encode( 'utf-8' ) .... return super( STRING, cls ).from_param( value ) .... >>>> STRING.from_param("titi") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 6, in from_param AttributeError: 'super' object has no attribute 'from_param' > > Thanks, > Mike > |
|
From: Mike C. F. <mcf...@vr...> - 2014-02-10 18:03:32
|
On 14-01-31 05:03 AM, rndblnch wrote: > Mike C. Fletcher <mcfletch <at> vrplumber.com> writes: ... > great, thanks! > > on my side, everything is ok on MacOSX 10.9.1, python2.7 & python3.3. > > the only regression i found now is when using pypy-2.1 or pypy3-2.1, but > they are related to pypy's implementation of ctypes, and i am not sure that > the pypy devs have any interest in fixing it as they are strongly supporting > cffi now. > i will report my finding to them anyway. > by the way, pypy has never sped any of my opengl code... Yeah, I'm seeing extremely poor performance under PyPy on what few demos I can get running without Pygame and Numpy. I fixed the GLUT error you pointed out, and registered some data-types that have different names in PyPy vs. CPython, and at least the core tests and a few GLUT Demos now run under PyPy. I'd say we're about ready for a 3.1.0b1 release for wider testing, so I'll do that the next time I get a few hours to work on PyOpenGL. Have fun, Mike -- ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com |
|
From: rndblnch <rnd...@gm...> - 2014-02-10 21:01:59
|
Mike C. Fletcher <mcfletch <at> vrplumber.com> writes: > > On 14-01-31 05:03 AM, rndblnch wrote: > > Mike C. Fletcher <mcfletch <at> vrplumber.com> writes: > ... > > great, thanks! > > > > on my side, everything is ok on MacOSX 10.9.1, python2.7 & python3.3. > > > > the only regression i found now is when using pypy-2.1 or pypy3-2.1, but > > they are related to pypy's implementation of ctypes, and i am not sure that > > the pypy devs have any interest in fixing it as they are strongly supporting > > cffi now. > > i will report my finding to them anyway. > > by the way, pypy has never sped any of my opengl code... > > Yeah, I'm seeing extremely poor performance under PyPy on what few demos > I can get running without Pygame and Numpy. I fixed the GLUT error you > pointed out, and registered some data-types that have different names in > PyPy vs. CPython, and at least the core tests and a few GLUT Demos now > run under PyPy. > > I'd say we're about ready for a 3.1.0b1 release for wider testing, so > I'll do that the next time I get a few hours to work on PyOpenGL. great! it works for me at r664 except when i pass tuples or lists instead of bytes to functions expecting a pointer to data under pypy (in my case, the last argument of glUniformMatrix3fv). the following patch fixes this (i guess that pypy does not throw the exact same exceptions than cpython): === modified file 'OpenGL/arrays/lists.py' --- OpenGL/arrays/lists.py 2014-01-02 20:27:57 +0000 +++ OpenGL/arrays/lists.py 2014-02-10 20:42:23 +0000 @@ -47,7 +47,7 @@ def from_param( self, instance, typeCode=None ): try: return ctypes.byref( instance ) - except TypeError as err: + except (TypeError, AttributeError) as err: array = self.asArray( instance, typeCode ) pp = ctypes.c_void_p( ctypes.addressof( array ) ) pp._temporary_array_ = (array,) and here the call stack i was getting before applying this patch: File "site-packages/PyOpenGL-3.1.0a4-py3.2.egg/OpenGL/platform/baseplatform.py", line 32, in __call__ return self.func( *args, **named ) File "lib_pypy/_ctypes/function.py", line 339, in __call__ args, kwargs) File "lib_pypy/_ctypes/function.py", line 561, in _convert_args keepalive, newarg, newargtype = self._conv_param(argtype, args[i]) File "/lib_pypy/_ctypes/function.py", line 455, in _conv_param arg = argtype.from_param(arg) File "/site-packages/PyOpenGL-3.1.0a4-py3.2.egg/OpenGL/arrays/arraydatatype.py", line 123, in from_param return cls.getHandler(value).from_param( value, cls.typeConstant ) File "/site-packages/PyOpenGL-3.1.0a4-py3.2.egg/OpenGL/arrays/lists.py", line 49, in from_param return ctypes.byref( instance ) File "/lib_pypy/_ctypes/basics.py", line 174, in byref return pointer(cdata) File "/lib_pypy/_ctypes/pointer.py", line 184, in pointer return POINTER(type(inst))(inst) File "/lib_pypy/_ctypes/pointer.py", line 178, in POINTER {'_type_': cls}) File "/lib_pypy/_ctypes/pointer.py", line 34, in __new__ self.set_type(obj, typedict['_type_']) File "/lib_pypy/_ctypes/pointer.py", line 73, in set_type self._ffiargtype = _ffi.types.Pointer(TP.get_ffi_argtype()) AttributeError: type object 'list' has no attribute 'get_ffi_argtype' renaud > > Have fun, > Mike > |
Thanks for helping keep SourceForge clean.
X