I'm trying to build MySQLdb on a shared webserver in my user account, and I can't seem to properly link to the mysql shared libraries. So far I have built both python2.4 and mysql-5.0.37 from source to ensure compiler compatibility using gcc 3.2.3 20030502 (Red Hat Linux 3.2.3-58).
mysql was built as follows (I don't know what some of these flags do, but I followed advice in the mysql docs):
export CFLAGS="-O3"
export CXX=gcc
export CXXFLAGS="-O3 -felide-constructors -fno-exceptions -fno-rtti"
./configure --prefix=$INSTALLDIR --enable-assembler --enable-thread-safe --enable-shared
make
make install
One comment at this point is that --enable-thread-safe failed to generate mysqlclient_r as advertised in the mysql docs, so I proceeded below with threadsafe = False in site.cfg
The build proceeds without error, but when I try to import:
% $HOME/bin/python2.4 -c "import MySQLdb"
Traceback (most recent call last):
File "<string>", line 1, in ?
File "build/bdist.linux-i686/egg/MySQLdb/init.py", line 19, in ?
File "build/bdist.linux-i686/egg/_mysql.py", line 7, in ?
File "build/bdist.linux-i686/egg/_mysql.py", line 6, in bootstrap
ImportError: libmysqlclient.so.15: cannot open shared object file: No such file or directory
libmysqlclient.so.15 is not on your loader path. Edit site.cfg, set static = True, and do a clean rebuild. That will probably work better than modifying the loader path in your case.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Andy - thanks for the reply. I rebuilt MySQLdb with the following settings:
embedded = False
threadsafe = False
static = True
but I still get an import error:
MySQL-python-1.2.2 % $HOME/bin/python2.4 -c "import MySQLdb"
/rc01/d94/ngh2/lib/python2.4/site-packages/MySQL_python-1.2.2-py2.4-linux-i686.egg/_mysql.py:3: UserWarning: Module _mysql was already imported from /rc01/d94/ngh2/lib/python2.4/site-packages/MySQL_python-1.2.2-py2.4-linux-i686.egg/_mysql.pyc, but /nfs/aesop11/rc01/d94/ngh2/build/MySQL-python-1.2.2 is being added to sys.path
Traceback (most recent call last):
File "<string>", line 1, in ?
File "MySQLdb/init.py", line 19, in ?
import _mysql
File "build/bdist.linux-i686/egg/_mysql.py", line 7, in ?
File "build/bdist.linux-i686/egg/_mysql.py", line 6, in bootstrap
ImportError: libmysqlclient.so.15: cannot open shared object file: No such file or directory
If I did want to change my loader path, how would I do it?
Thanks again,
Noah
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Like I said, you need to do a clean build. You didn't recompile _mysql.so if you are getting that error. Run setup.py clean or remove the build directory and try again.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Andy - Before rebuilding I was careful to remove the MySQL-python-1.2.2 directory, re-expand the gz archive, and remove the installed python egg
rm ~/lib/python2.4/site-packages/My*.egg
Is there something else I need to remove to ensure that I'm starting from scratch? I'm also quite sure that the output from setup.py contained the calls to gcc that I put in my original post - so I'm pretty sure I'm recompiling _mysql.so
Any ideas? Any other info I can provide?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Well, if you deleted the entire directory (you only needed to remove the build directory), did you then edit site.cfg and set static = True before building?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
My understanding of this is that the -L/rc01/d94/ngh2/mysql-5.0.37-linux/lib/mysql -lmysqlclient flags were preventing static linking to libmysqlclient.a
I'm trying to build MySQLdb on a shared webserver in my user account, and I can't seem to properly link to the mysql shared libraries. So far I have built both python2.4 and mysql-5.0.37 from source to ensure compiler compatibility using gcc 3.2.3 20030502 (Red Hat Linux 3.2.3-58).
mysql was built as follows (I don't know what some of these flags do, but I followed advice in the mysql docs):
export INSTALLDIR=$HOME/mysql-5.0.37-linux
mkdir $INSTALLDIR
see http://dev.mysql.com/doc/refman/5.0/en/installing-source.html
export CFLAGS="-O3"
export CXX=gcc
export CXXFLAGS="-O3 -felide-constructors -fno-exceptions -fno-rtti"
./configure --prefix=$INSTALLDIR --enable-assembler --enable-thread-safe --enable-shared
make
make install
One comment at this point is that --enable-thread-safe failed to generate mysqlclient_r as advertised in the mysql docs, so I proceeded below with threadsafe = False in site.cfg
Now I build MySQLdb:
setup.py must be able to see mysql_config
export PATH=$HOME/mysql-5.0.37-linux/bin:$PATH
export INSTALLDIR=$HOME/lib/python2.4/site-packages/
export PYTHONPATH=$INSTALLDIR:$PYTHONPATH
mkdir -p $INSTALLDIR
$HOME/bin/python2.4 setup.py install --prefix=$HOME
The build proceeds without error, but when I try to import:
% $HOME/bin/python2.4 -c "import MySQLdb"
Traceback (most recent call last):
File "<string>", line 1, in ?
File "build/bdist.linux-i686/egg/MySQLdb/init.py", line 19, in ?
File "build/bdist.linux-i686/egg/_mysql.py", line 7, in ?
File "build/bdist.linux-i686/egg/_mysql.py", line 6, in bootstrap
ImportError: libmysqlclient.so.15: cannot open shared object file: No such file or directory
I clearly have built libmysqlclient.so.15:
~ % find ~/mysql-5.0.37-linux -name "libmysqlclient*"
/rc01/d94/ngh2/mysql-5.0.37-linux/lib/mysql/libmysqlclient.so.15.0.0
/rc01/d94/ngh2/mysql-5.0.37-linux/lib/mysql/libmysqlclient.so.15
/rc01/d94/ngh2/mysql-5.0.37-linux/lib/mysql/libmysqlclient.so
/rc01/d94/ngh2/mysql-5.0.37-linux/lib/mysql/libmysqlclient.la
/rc01/d94/ngh2/mysql-5.0.37-linux/lib/mysql/libmysqlclient.a
And it looks like this directory was at least mentioned in the compile code (partial output from setup.py install)
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -Dversion_info=(1,2,2,'final',0) -Dversion=1.2.2 -I/rc01/d94/ngh2/mysql-5.0.37-linux/include/mysql -I/rc01/d94/ngh2/include/python2.4 -c _mysql.c -o build/temp.linux-i686-2.4/_mysql.o
gcc -pthread -shared build/temp.linux-i686-2.4/_mysql.o -L/rc01/d94/ngh2/mysql-5.0.37-linux/lib/mysql -lmysqlclient -lz -lcrypt -lnsl -lm -o build/lib.linux-i686-2.4/_mysql.so
Yet clearly something didn't work correctly:
~ % ldd /rc01/d94/ngh2/build/MySQL-python-1.2.2/build/lib.linux-i686-2.4/_mysql.so
libmysqlclient.so.15 => not found
libz.so.1 => /usr/lib/libz.so.1 (0x0025b000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0x00292000)
libnsl.so.1 => /lib/libnsl.so.1 (0x00bb2000)
libm.so.6 => /lib/tls/libm.so.6 (0x00162000)
libpthread.so.0 => /lib/tls/libpthread.so.0 (0x0013c000)
libc.so.6 => /lib/tls/libc.so.6 (0x00de8000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x008b7000)
Can someone help me put this together? I'm very much at the limit of my understanding of the compile process at this point.
Thanks a lot for any ideas,
Noah
libmysqlclient.so.15 is not on your loader path. Edit site.cfg, set static = True, and do a clean rebuild. That will probably work better than modifying the loader path in your case.
Andy - thanks for the reply. I rebuilt MySQLdb with the following settings:
embedded = False
threadsafe = False
static = True
but I still get an import error:
MySQL-python-1.2.2 % $HOME/bin/python2.4 -c "import MySQLdb"
/rc01/d94/ngh2/lib/python2.4/site-packages/MySQL_python-1.2.2-py2.4-linux-i686.egg/_mysql.py:3: UserWarning: Module _mysql was already imported from /rc01/d94/ngh2/lib/python2.4/site-packages/MySQL_python-1.2.2-py2.4-linux-i686.egg/_mysql.pyc, but /nfs/aesop11/rc01/d94/ngh2/build/MySQL-python-1.2.2 is being added to sys.path
Traceback (most recent call last):
File "<string>", line 1, in ?
File "MySQLdb/init.py", line 19, in ?
import _mysql
File "build/bdist.linux-i686/egg/_mysql.py", line 7, in ?
File "build/bdist.linux-i686/egg/_mysql.py", line 6, in bootstrap
ImportError: libmysqlclient.so.15: cannot open shared object file: No such file or directory
If I did want to change my loader path, how would I do it?
Thanks again,
Noah
Like I said, you need to do a clean build. You didn't recompile _mysql.so if you are getting that error. Run setup.py clean or remove the build directory and try again.
Andy - Before rebuilding I was careful to remove the MySQL-python-1.2.2 directory, re-expand the gz archive, and remove the installed python egg
rm ~/lib/python2.4/site-packages/My*.egg
Is there something else I need to remove to ensure that I'm starting from scratch? I'm also quite sure that the output from setup.py contained the calls to gcc that I put in my original post - so I'm pretty sure I'm recompiling _mysql.so
Any ideas? Any other info I can provide?
Well, if you deleted the entire directory (you only needed to remove the build directory), did you then edit site.cfg and set static = True before building?
Yes. I know this because if I don't also set threadsafe = False (default is True), compilation fails.
If you set static = True, then ldd should not show any references to libmysqlclient.so.15.
Post the output of: setup.py build_ext
You'll need to clean out the build directory first for this to be meaningful.
After deleting and re-expanding MySQL-python-1.2.2/
modify site.cfg:
embedded = False
threadsafe = False
static = True
export PATH=$HOME/mysql-5.0.37-linux/bin:$PATH
export INSTALLDIR=$HOME/lib/python2.4/site-packages/
export PYTHONPATH=$INSTALLDIR:$PYTHONPATH
mkdir -p $INSTALLDIR
$HOME/bin/python2.4 setup.py build_ext
Output:
running build_ext
building '_mysql' extension
creating build
creating build/temp.linux-i686-2.4
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -Dversion_info=(1,2,2,'final',0) -Dversion=1.2.2 -I/rc01/d94/ngh2/mysql-5.0.37-linux/include/mysql -I/rc01/d94/ngh2/include/python2.4 -c _mysql.c -o build/temp.linux-i686-2.4/_mysql.o
creating build/lib.linux-i686-2.4
gcc -pthread -shared build/temp.linux-i686-2.4/_mysql.o /rc01/d94/ngh2/mysql-5.0.37-linux/lib/mysql/libmysqlclient.a -L/rc01/d94/ngh2/mysql-5.0.37-linux/lib/mysql -lmysqlclient -lz -lcrypt -lnsl -lm -o build/lib.linux-i686-2.4/_mysql.so
check _mysql.so:
MySQL-python-1.2.2 % ldd /rc01/d94/ngh2/build/MySQL-python-1.2.2/build/lib.linux-i686-2.4/_mysql.so
libmysqlclient.so.15 => not found
libz.so.1 => /usr/lib/libz.so.1 (0x00544000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0x005df000)
libnsl.so.1 => /lib/libnsl.so.1 (0x00171000)
libm.so.6 => /lib/tls/libm.so.6 (0x00b84000)
libpthread.so.0 => /lib/tls/libpthread.so.0 (0x00c4a000)
libc.so.6 => /lib/tls/libc.so.6 (0x00185000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00c89000)
Thanks,
Noah
For anyone following this thread, here's how I managed to solve this linking problem (after learning a bit about static versus dynamic linking - I found the key in this reference: http://www.network-theory.co.uk/docs/gccintro/gccintro_25.html ):
After performing the steps described above, I recompiled _mysql.so by hand.
Original output of setup.py:
gcc -pthread -shared build/temp.linux-i686-2.4/_mysql.o /rc01/d94/ngh2/mysql-5.0.37-linux/lib/mysql/libmysqlclient.a -L/rc01/d94/ngh2/mysql-5.0.37-linux/lib/mysql -lmysqlclient -lz -lcrypt -lnsl -lm -o build/lib.linux-i686-2.4/_mysql.so
Changed to:
gcc -pthread -shared build/temp.linux-i686-2.4/_mysql.o /rc01/d94/ngh2/mysql-5.0.37-linux/lib/mysql/libmysqlclient.a -lz -lcrypt -lnsl -lm -o build/lib.linux-i686-2.4/_mysql.so
My understanding of this is that the -L/rc01/d94/ngh2/mysql-5.0.37-linux/lib/mysql -lmysqlclient flags were preventing static linking to libmysqlclient.a
Now I get:
% ldd build/lib.linux-i686-2.4/_mysql.so
libz.so.1 => /usr/lib/libz.so.1 (0x00df9000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0x009ae000)
libnsl.so.1 => /lib/libnsl.so.1 (0x00a5b000)
libm.so.6 => /lib/tls/libm.so.6 (0x00855000)
libpthread.so.0 => /lib/tls/libpthread.so.0 (0x00228000)
libc.so.6 => /lib/tls/libc.so.6 (0x00238000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00c6e000)
And no problems on import.
I'd be curious to know if this is a bug in the install scripts or if I have overlooked the proper way to accomplish the same thing.
There's probably an issue with static builds; I'll try to look at it this weekend.