Debian Based and import numpy

[tom]

I have a program which uses PythonQt and having some problem when doing a import numpy
Here is what I get.

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/numpy/__init__.py", line 180, in <module>
    from . import add_newdocs
  File "/usr/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/usr/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 8, in <module>
    from .type_check import *
  File "/usr/lib/python2.7/dist-packages/numpy/lib/type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "/usr/lib/python2.7/dist-packages/numpy/core/__init__.py", line 14, in <module>
    from . import multiarray
ImportError: /usr/lib/python2.7/dist-packages/numpy/core/multiarray.x86_64-linux-gnu.so: undefined symbol: PyExc_SystemError

Digging around, I found that on certain linux (mint in my case, but probably also Ubuntu and Debian) the python library is not loaded with RTLD_LAZY | RTLD_GLOBAL so that all symbols defined there are not accessible from a library loaded at runtime.

I fixed this by adding #include <dlfcn.h> and dlopen("libpython2.7.so", RTLD_LAZY | RTLD_GLOBAL); just before the PythonQt::init(PythonQt::IgnoreSiteModule|PythonQt::RedirectStdOut);

I don't like this since my program is cross platform and it force me to use of #ifdef and moreover this works for a PythonQt compiled with python 2.7 and not for 3.x

Do you ever encouter this issue?
Do you have any advice for a better fix?

many thanks

[David Brooks]

My workaround for this was to do the following:

// We need to know the state of Q_OS_LINUX
#include <Qt>

// The Python header must be included before <QObject> because of name clashes
#ifdef Q_OS_LINUX
    #include "Python.h"
#endif

// ...

int main(int argc, char *argv[])
{
#ifdef Q_OS_LINUX
    // The Python library needs linking directly to the executable as otherwise
    // Python extension shared objects can't find symbols in the library. This is
    // because the lookup scope changes for DSOs that are are loaded using dlopen()
    // (see https://www.akkadia.org/drepper/dsohowto.pdf for details)

    // This is sufficient to ensure the library is linked to the executable
    Py_NoUserSiteDirectory = 1;
#endif

// ...

}

[tom]

Thanks David,
I finally found out what caused the error.

I have PyhonQt in my application loaded at runtime as plugin.

The way I was loading this plugin (which in turns load the PythonQt library and then the Python library) by using (a subclass of) QPluginLoader was hiding the symbols in the Python library.

I had to modify by adding

setLoadHints(QLibrary::ResolveAllSymbolsHint | QLibrary::ExportExternalSymbolsHint);

which I leave here in case someone need.