numpy-discussion

[Numpy-discussion] Threading, multi-processors, and Numeric Python

[Joe V. A. <van...@at...>]

I've written Numeric Python code (with Python 1.5.2) to analyze weather
radar data.  In an attempt to speed up this code, I used threads to
perform some of the computations.  

I'm running on a dual processor Linux machine running 2.4.1 with SMP
enabled.  I'm using Numeric -17.3.0 with Python 1.5.2

When I run the threaded code, and monitor the system with 'top', 1
processor spends much of its time idle, and I rarely see two copies of
my 'compute' thread executing.  Each thread is computing its results
from different arrays.  However, all arrays are referenced from the same
dictionary.

Any ideas on how to get both threads computing at the same time?

Thanks for your help!

-- 
Joe VanAndel  	          
National Center for Atmospheric Research
http://www.atd.ucar.edu/~vanandel/
Internet: van...@uc...

Re: [Numpy-discussion] Threading, multi-processors, and Numeric Python

[Joe V. A. <van...@at...>]

OK, I guess I've just discovered the answer to my own question.  [Don't
you just hate that! :-) ]


My C++ extensions that perform the real calculations needed to be
modified to support multiple threads.  By adding the
Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS macros, the Python
interpreter knew it was safe to allow other threads to execute during
computations, just like it allows other threads to execute during I/O.

 To accomodate the Python global thread lock, I needed to change my code
as follows:

my_func()
{

	/* python calls */

        Py_BEGIN_ALLOW_THREADS

        /* computations that don't use python API */


	Py_END_ALLOW_THREADS

	/* python API calls */
}

Now, I can see both processors being used.

-- 
Joe VanAndel  	          
National Center for Atmospheric Research
http://www.atd.ucar.edu/~vanandel/
Internet: van...@uc...