[PyWX] Py_Initialize

[christopher m. <cmu...@av...>]

Titus et.al,

I managed to solve the 'unresolved symbol 'Py_Initialize' ' failure
on loading nspywx.so.  Aolserver-3.4.2 and Python2.2 are good to go.

Titus suggested that it sounded like my python-installation wasn't
configured for dynamic loading, ie. (AKAIK) no 'libpython.so'
He was right.

Documentation is a little scarce on this topic, so I'll share the
war-story and prime the google pump.

Note: I have no formal CS training.  All of my knowledge is based
upon building a lot of software packages ( the linuxfromscratch.org school )
Much of the following may be redundant or just plain wrong.
You have been warned  ;)


First 'libpython.so' 

I found one post and reply on the Python list regarding the creation of
a python '.so'.  To paraphrase:

	mkdir tmp; cd tmp
	ar xf /usr/lib/python2.2/config/libpython2.2a
	cd ..
	
	# a '-v' switch to 'ar' or a subsequent listing of tmp/ will
	#  show all the '.o' files that comprise the libpython2.2a library

	gcc -shared -fPIC -o libpython2.2so tmp/*.o

This creates your libpython($VERSION).so

I am not sure what the canonical home for this library is,
in $PYTHONHOME/config alongside libpython2.2.a, or in /usr/lib:/usr/local/lib,
but for purposes of aolserver integration this distinction is less important
since the 'nsd8x' daemon can ( and therefore IMHO should ) be run in a
chroot'd environment.

It took me a little while to have confidence in this new libpython.so
because of linking issues arising from 'chroot'ing the aolserver daemon. 

chroot ( for anyone unfamiliar ) is a system call ( or the command line program
that invokes it ) available only to root, that exec's another process 'rooted'
to some arbitrary point in the file system, ie: 
   If aolserver is run chrooted to '/usr/local/aolserver',
   '/usr/local/aolserver/lib' becomes '/lib',
   '/usr/local/aolserver/dev' becomes '/dev',
	etc.
	and the paths in config.tcl need to reflect this.
This new 'root' is inviolable and chroot's child process has no access to
any part of the filesystem not under it.  Combined with declination of privilages
, chroot is network daemon best-practice, but it has dynamic-linking implications.
Any shared libraries or device files required by the chroot'ed app must be made
available under the chroot.

 
I began populating /usr/local/aolserver/lib with all those libraries I
felt that a Python interpreter would need, largely based upon the ouput
of `ldd /usr/bin/python`:

	libdl.so.2 , libpthread.so.0, libutil.so.1, libm.so.6, libc.so.6, ld-linux.so.2

	as well as my new libpython2.2.so.

This was incomplete however, as I still had one unresolved symbol:

	'_._9streambuf'

`grep _._9streambuf /usr/lib/*` points us to lidstdc++.so

With the inclusion of libstdc++.so under chroot/lib this problem persisted
however.  The solution may have been simply to create a chroot/usr/lib and place
libstdc++.so there as '/usr/lib/libstdc++.so' was the path Pywx was built against
but I am not sure as I elected to rebuild nspywx.so with my Makefile modified
as follows:

LIBS = -L/usr/local/aolserver/lib -lpthread -ldl -lpython2.2 -lutil -lstdc++

This solved all unresolved symbols issues but the newly loaded nspywx.so
barked about not being able to find the python libraries:

[04/Aug/2002:22:47:33][24059.1024][-main-] Notice: modload: loading '/bin/nspywx.so'
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
'import site' failed; use -v for traceback

Doh!

This makes sense.  The python libs are not visible within the chroot.


( simple-fix )

	mkdir -p /usr/local/aolserver/usr/lib
	cp -pR /usr/lib/python2.2 /usr/local/aolserver/usr/lib

All by itself, copying the python libs into the chroot resulted in the same
error.  Invoking aolserver ( from BASH) as follows:

	cd /usr/local/aolserver

	PYTHONHOME='/usr/lib/python2.2' bin/nsd8x -u aolserver -g aolserver \
		-r /usr/local/aolserver -t config.tcl

 takes care of the library errors, although the following error msg:

'import site' failed; use -v for traceback
	
persists.

( 
	I am new to Python, and PYTHONHOME / PYTHONPATH errors are a major frustation.
	I do not know if these settings can be modified post-compilation.
	I have a LFS-build with a nonstandard directory structure -- /usr/lib
	is actually /lib/usr, /usr/sbin is /sbin/usr, /usr/bin is /bin/usr,
	/var/qmail/bin is /bin/qmail etc. and python installed as /bin/python
	with libraries under the decidedly non-standard /lib/python
	-- The intention being ( with separate
	partitions for boot,var,tmp,& usr ) to create a small  '/' partition
	with good geography nestled between a tiny boot partition and the swap 
	partition at the front of the disk, containing all the executable binaries and libs
	mounted READONLY with all remaining partitions mounted noexec,nodev,nosuid.
	This seems like a good idea to me ( although only 100% effective if the root
	partition is mounted on a device that is physically read-only )
	but it leads to a lot of extra work when building software packages,
	and in the case of Python ( where I am still struggling with the
	PYTHONHOME/PYTHONPATH issues ) runtime failures.
	But I digress.
)

This is where I am now.  I am not sure if the 'import site' error is fatal
( see above for declaration of ignorance.)  I have only recently solved
build & linking issues with nspywx.so, and my aolserver displays the '*.py'
example scripts as raw source-code rather than executing them.  I haven't
yet spent a lot of time poring over config.tcl directives necesary to map
these files to the python interpreter, so I am not too concerned about that yet.


cheers,

mulc