pydoop 0.8.1, hadoop 1.0.4 _pipes_1_0_4.so undefined symbol

Help
Al Piszcz
2013-02-01
2013-02-06
  • Al Piszcz

    Al Piszcz - 2013-02-01

    DEBIAN 12.10, pydoop-0.8.1, python 2.7.3, hadoop-1.0.4

    The undefined symbol: BIO_s_mem (SSL) is being thrown as an import error.

    Python 2.7.3 (default, Sep 26 2012, 21:51:14)
    [GCC 4.7.2] on linux2
    Type "help", "copyright", "credits" or "license" for more information.

    import pydoop.pipes
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/usr/local/lib/python2.7/dist-packages/pydoop/pipes.py", line 29, in <module>
    pp = pydoop.import_version_specific_module('_pipes')
    File "/usr/local/lib/python2.7/dist-packages/pydoop/init.py", line 107, in import_vers
    ion_specific_module
    return import_module(complete_mod_name(name))
    File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module
    import(name)
    ImportError: /usr/local/lib/python2.7/dist-packages/pydoop/_pipes_1_0_4.so: undefined symbol
    : BIO_s_mem

    When the ssl library is set in LD_PRELOAD pydoop.pipes can be loaded

    $ export LD_PRELOAD=/lib/x86_64-linux-gnu/libssl.so.1.0.0
    $ python
    Python 2.7.3 (default, Sep 26 2012, 21:51:14)
    [GCC 4.7.2] on linux2
    Type "help", "copyright", "credits" or "license" for more information.

    import pydoop.pipes

    The result is when the tasks are mapped to the nodes they hang for the 10 minute time due to this issue. Is there a better way to get the SSL library besides LD_PRELOAD?

    Thanks

     
    • Al Piszcz

      Al Piszcz - 2013-02-01

      solved:

      1 setup.py is missing the crypto library during build, added crypto.

      2 logout of system and back in before testing

      WAS
      libraries.append("ssl")

      NOW
      libraries.append("ssl")
      libraries.append("crypto")

      Python 2.7.3 (default, Sep 26 2012, 21:51:14)
      [GCC 4.7.2] on linux2
      Type "help", "copyright", "credits" or "license" for more information.

      import pydoop.pipes

       
  • Simone Leo

    Simone Leo - 2013-02-04

    Hello,

    Linking to libssl is enough, Pydoop does not need libcrypto. The problem is that your system, at runtime, finds libcrypto but not libssl.

    LD_PRELOAD is mostly used as a hack to override functions in other libraries. Modifying LD_LIBRARY_PATH should be enough to solve the problem. But why isn't /lib/x86_64-linux-gnu auto-searched for runtime linking on your system?

    Simone

     
    Last edit: Simone Leo 2013-02-04
    • Al Piszcz

      Al Piszcz - 2013-02-06

      Thanks for the reply.

      The environment is UNBUNTO 12.04
      I reviewed the Python 2.73 setup.py they appear to include crypto with ssl
      in, it may be a different reason.

      setup.py
      exts.append( Extension('_ssl', ['_ssl.c'],
      include_dirs = ssl_incs,
      library_dirs = ssl_libs,
      libraries = ['ssl', 'crypto'],
      depends = ['socketmodule.h']), )

       

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks