[Pympi-users] class MyServer(mpi.Server):

[Julian C. <rjc...@cs...>]

Getting back to the subject of the server, originally discussed under
"Reposted: Web Services and pyMPI:"

Pat Miller wrote:

Here it sounds like you want to have a distributed replacement for what is
typically done with threads. E.g. rank 0 takes requests and passes them
off to other workers for completion.

At the end of his code sample he added:

A more complete version would need allow the clients themselves to be
parallel
 (on a sub-communicator) so you could both process requests in parallel and
 process each one individually in parallel.

The latter is more like what I'm thinking about (clients themselves are
parallel on a sub-communicator). To make this work, without a lot of
complexity, I think we would have to start multiple [mostly idle] p4 devices
per cpu, as a pool and use them as needed:

example:
On 10 CPU machine, instead starting 10 p4 devices, start 51 i.e. one for the
server and 5 pools of 10 cpu's. Technically this could handle 5 simultaneous
requests. After 5 simultaneous requests, it would have to queue or reject -
assuming that MPI has no ability to dynamically start more slaves (MPI-2 ?).

The advantage of this is that performance degrades slowly and the operating
system handles the multi-tasking of all jobs. There are, however a lot of
idle p4 devices taking up sockets, but that is not a major problem.

The main task of the server is managing pools i.e. decrementing available
pools as requests come in and incrementing again, when the result is
returned. I think that the classic Globus grid implementations still handle
this by placing executables into a batch queue, which is very
unsophisticated. Pat would probably know better.

In any case, I think the pool approach would actually simplify the server
design over the original that Pat created.

Julian Cook