Menu

#138 glut apps segfault due to use of unchecked null pointer

closed-fixed
nobody
nuisance (37)
5
2011-01-23
2010-09-03
No

Currently certain mesa drivers (namely intel_dri) fail to return propper visualInfo with glXGetVisualFromFBConfig() as used within fgOpenWindow().
This visualInfo pointer is later dereferenced unchecked to create the Colormap and causes a segfault.

While this is not freegluts fault, using the value unchecked is fatal and causing the application a segfault (which currently hits any application on intel drifb X11 hardware.

Attached is a small patch that gives the user a clue what happened and exits the application a bit more gracefully.

Discussion

  • Andreas Arens

    Andreas Arens - 2010-09-03

    patch to check driver return

     
  • Andreas Arens

    Andreas Arens - 2010-09-03

    Further investigation showed that the intel driver just returns the array of FBConfigs in an order where the first matching config does not support windowed mode. Since fgChooseFBConfig() within freeglut returns the complete array of matching configs this is relatively easy to fix. All it takes is to pass the number of returned configs to the caller, and then have the call iterate over the array to find a matching FBConfig that supports windowed mode an therefor returns a visualInfo.
    The following minimal inversive patch does just this.
    With the patch applied freeglut becomes usable with the intel driver (xorg-1.8.2, intel git head, mesa-7.8.2)

     
  • Andreas Arens

    Andreas Arens - 2010-09-03

    patch to iterate over FBConfigs

     
  • John F. Fay

    John F. Fay - 2011-01-23
    • status: open --> closed-fixed
     
  • John F. Fay

    John F. Fay - 2011-01-23

    Fixed at changeset 884.

     

Log in to post a comment.