[Pytables-users] Table.where() hangs when called from subprocess

[Mario J. <mj...@cf...>]

Hi everyone,
	I'm evaluating PyTables as a backend for data storage in large
astronomical surveys. So far the results have been fantastic, except for
the following issue: if a table is opened and iterated on from within a
subprocess instantiated with multiprocessing.Process, the program hangs.

This test case reproduces the problem with pytables 2.2 on Python 2.6
and 2.7:

===========================
#!/usr/bin/env python

import tables
import multiprocessing as mp

def _worker(qout = None):
	fn = "testcase.h5"
	fp = tables.openFile(fn)
	print "About to load: ", fn
	rows = fp.root.catalog.where('(ra > 0)')
	print "Got the iterator, about to iterate"
	row = next(rows)
	print "Succeeded in one iteration\n"
	fp.close()

	if qout is not None:
		qout.put("Done")

if __name__ == "__main__":
	print "**** Running from main process:"
	_worker()

	print "**** Running from subprocess:"
	qout = mp.Queue()
	ps = mp.Process(target=_worker, args=(qout,))
	ps.daemon = True
	ps.start()

	print qout.get();
===========================

The output:
============================
[mjuric@nebel skysurvey]$ ./db.py
**** Running from main process:
About to load:  testcase.h5
Got the iterator, about to iterate
Succeeded in one iteration

**** Running from subprocess:
About to load:  testcase.h5
Got the iterator, about to iterate
[[[ ... and here the process just hangs ]]]
============================

The testcase.h5 file can be found on
http://nebel.rc.fas.harvard.edu/mjuric/pytables/ , but I suspect the
problem is not file related (it happens with nearly every one I tried).

Is this a known issue (and is there a workaround), or am I doing
something wrong?

Regards,
-- 
Mario Juric,
Hubble Fellow,  Harvard-Smithsonian Center for Astrophysics
Web   :  http://www.sns.ias.edu/~mjuric/
Phone :  +1 617 744-9003     PGP: ~mjuric/crypto/public.key

E-mail reading/answering policy:  http://majuric.org/email/