Thread: [BProc] Clustermatic and MPI

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Hello,

Can I use mpich with clustermatic?.

Carlos.

On Sun, 16 Dec 2001, Carlos J. Garcia Orellana wrote:

> Hello,
>=20
> Can I use mpich with clustermatic?.

Clustermatic uses a setup very similar to Scyld's, and Scyld ships a
version of mpich modified to work nicely with BProc.  You might need to
make some miner changes as Scyld's mpich was modified for BProc 2.2 and
clustermatic uses BProc 3 (which has a slightly different API), but
other than that Scyld's MPICH should compile and work on a clustermatic
system.

If you want a supported and out of the box system using BProc, you can
also look at Scyld Beowulf,  http://www.scyld.com/

On 20011216 Jag wrote:
>On Sun, 16 Dec 2001, Carlos J. Garcia Orellana wrote:
>
>> Hello,
>> 
>> Can I use mpich with clustermatic?.
>
>Clustermatic uses a setup very similar to Scyld's, and Scyld ships a
>version of mpich modified to work nicely with BProc.  You might need to
>make some miner changes as Scyld's mpich was modified for BProc 2.2 and
>clustermatic uses BProc 3 (which has a slightly different API), but
>other than that Scyld's MPICH should compile and work on a clustermatic
>system.
>
>If you want a supported and out of the box system using BProc, you can
>also look at Scyld Beowulf,  http://www.scyld.com/
>

The chages introduced in Scyld MPI are now in standard MPICH. So get
the newest mpich from
http://www-unix.mcs.anl.gov/mpi/mpich
and build it.

In other words, standard mpich has now support for bproc.

-- 
J.A. Magallon                           #  Let the source be with you...        
mailto:jam...@ab...
Mandrake Linux release 8.2 (Cooker) for i586
Linux werewolf 2.4.17-rc1-beo #1 SMP Fri Dec 14 09:58:53 CET 2001 i686

Yes, but unfortunatley it is specific to Scyld's use of Bproc. They have
hacked in dependancies for beomap and beostatus. To use mpich requires you
to use this programs instead of another resource manager.

Nic

...snip...
>
> The chages introduced in Scyld MPI are now in standard MPICH. So get
> the newest mpich from
> http://www-unix.mcs.anl.gov/mpi/mpich
> and build it.
>
> In other words, standard mpich has now support for bproc.
>
>

On Sun, Dec 16, 2001 at 05:21:39PM -0500, Nicholas Henke wrote:
> Yes, but unfortunatley it is specific to Scyld's use of Bproc. They have
> hacked in dependancies for beomap and beostatus. To use mpich requires you
> to use this programs instead of another resource manager.

We saw that here and ended up making our own little modification.
Attached below is the MPICH patch.  The down side is you need a
special MPI run to use this.  Unfortunately since that's a separate
piece of code that I wrote from scratch here, there's a procedure to
do through to release that.

It's a very simple program though.  Somebody could rewrite it a LOT
faster than I can get it released if they're feeling impatient.

The patch just creates a new "external execer" facility.  For the
program "app" and -np 4, mpirun would fork and bproc_execmove the
following:

rank 0: app -p4execer 0 4 n-1 45541 ;n5,0;n6,1;n7,1;n10,1
rank 1: app -p4execer 1 4 n5 41922
rank 2: app -p4execer 2 4 n5 41922
rank 3: app -p4execer 3 4 n5 41922

-p4execer is the magic argument and it works like this:

for rank 0:
-p4execer rank jobsize mpirunhost mpirunport procgroup

for rank 1+:
-p4execer rank jobsize rank0host rank0port

The reason rank 0 is special is because it is the job that all the
others must connect to in MPI_Init.  In order to do that the others
must know what host and port rank 0 is on.  mpirun won't know what
port to tell the others unless rank 0 tells it.  That's why rank zero
connects to mpirun and sends its port number.  Then mpirun can start
all the other jobs with approprate arguments.

The format of the process group argument is:

;host0,0;host1,1;host2,1;host3,1

You could just wait for me to get our simple mpirun released but it
probably wont be for a while since I probably can't do it before xmas
and the lab is closed for a week then.

- Erik

Thanks!! This makes much more sense... and it works nicely without having
to execute from the master node. I am taking a month off for semester
break, so I will check for your mpirun script when I get back.

Thanks a ton-- keep up the good work. It is nice to see bproc based stuff
without seeing it based specifically on Scyld.

Nic

On Mon, 17 Dec 2001, Erik Arjan Hendriks wrote:

[SNIP]
>
> We saw that here and ended up making our own little modification.
> Attached below is the MPICH patch.  The down side is you need a
> special MPI run to use this.  Unfortunately since that's a separate
> piece of code that I wrote from scratch here, there's a procedure to
> do through to release that.
>
> It's a very simple program though.  Somebody could rewrite it a LOT
> faster than I can get it released if they're feeling impatient.
>
> The patch just creates a new "external execer" facility.  For the
> program "app" and -np 4, mpirun would fork and bproc_execmove the
> following:
>
> rank 0: app -p4execer 0 4 n-1 45541 ;n5,0;n6,1;n7,1;n10,1
> rank 1: app -p4execer 1 4 n5 41922
> rank 2: app -p4execer 2 4 n5 41922
> rank 3: app -p4execer 3 4 n5 41922
>
> -p4execer is the magic argument and it works like this:
>
> for rank 0:
> -p4execer rank jobsize mpirunhost mpirunport procgroup
>
> for rank 1+:
> -p4execer rank jobsize rank0host rank0port
>
> The reason rank 0 is special is because it is the job that all the
> others must connect to in MPI_Init.  In order to do that the others
> must know what host and port rank 0 is on.  mpirun won't know what
> port to tell the others unless rank 0 tells it.  That's why rank zero
> connects to mpirun and sends its port number.  Then mpirun can start
> all the other jobs with approprate arguments.
>
> The format of the process group argument is:
>
> ;host0,0;host1,1;host2,1;host3,1
>
>
> You could just wait for me to get our simple mpirun released but it
> probably wont be for a while since I probably can't do it before xmas
> and the lab is closed for a week then.
>
> - Erik
>

-- 
Nicholas Henke
Undergraduate - SEAS '02
Liniac Project - University of Pennsylvania
http://clubmask.sourceforge.net
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
work: 215-873-5149
cell/home: 215-681-2705
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
There's nothing like good food, good beer, and a bad girl.

One question more: Is necesary to have libbeostat and libbeomap to use mpi
with Erik's patch?

Last weekend, I've modificated libbeostat to run with 2.4.13 kernel and now
I have working a mpi 1.2.2 build as rpm package (using the spec file
provided by Scyld in utils dir of mpich 1.2.2 distribution). I looks work
fine.

Are there any problem in using libbeostat?.

Thanks a lot:

Carlos.

----- Original Message -----
From: "Nicholas Henke" <he...@se...>
To: "Erik Arjan Hendriks" <er...@he...>
Cc: <bpr...@so...>
Sent: Monday, December 17, 2001 11:11 PM
Subject: Re: [BProc] Clustermatic and MPI

> Thanks!! This makes much more sense... and it works nicely without having
> to execute from the master node. I am taking a month off for semester
> break, so I will check for your mpirun script when I get back.
>
> Thanks a ton-- keep up the good work. It is nice to see bproc based stuff
> without seeing it based specifically on Scyld.
>
> Nic
>
> On Mon, 17 Dec 2001, Erik Arjan Hendriks wrote:
>
> [SNIP]
> >
> > We saw that here and ended up making our own little modification.
> > Attached below is the MPICH patch.  The down side is you need a
> > special MPI run to use this.  Unfortunately since that's a separate
> > piece of code that I wrote from scratch here, there's a procedure to
> > do through to release that.
> >
> > It's a very simple program though.  Somebody could rewrite it a LOT
> > faster than I can get it released if they're feeling impatient.
> >
> > The patch just creates a new "external execer" facility.  For the
> > program "app" and -np 4, mpirun would fork and bproc_execmove the
> > following:
> >
> > rank 0: app -p4execer 0 4 n-1 45541 ;n5,0;n6,1;n7,1;n10,1
> > rank 1: app -p4execer 1 4 n5 41922
> > rank 2: app -p4execer 2 4 n5 41922
> > rank 3: app -p4execer 3 4 n5 41922
> >
> > -p4execer is the magic argument and it works like this:
> >
> > for rank 0:
> > -p4execer rank jobsize mpirunhost mpirunport procgroup
> >
> > for rank 1+:
> > -p4execer rank jobsize rank0host rank0port
> >
> > The reason rank 0 is special is because it is the job that all the
> > others must connect to in MPI_Init.  In order to do that the others
> > must know what host and port rank 0 is on.  mpirun won't know what
> > port to tell the others unless rank 0 tells it.  That's why rank zero
> > connects to mpirun and sends its port number.  Then mpirun can start
> > all the other jobs with approprate arguments.
> >
> > The format of the process group argument is:
> >
> > ;host0,0;host1,1;host2,1;host3,1
> >
> >
> > You could just wait for me to get our simple mpirun released but it
> > probably wont be for a while since I probably can't do it before xmas
> > and the lab is closed for a week then.
> >
> > - Erik
> >
>
> --
> Nicholas Henke
> Undergraduate - SEAS '02
> Liniac Project - University of Pennsylvania
> http://clubmask.sourceforge.net
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> work: 215-873-5149
> cell/home: 215-681-2705
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> There's nothing like good food, good beer, and a bad girl.
>
>
>
> _______________________________________________
> BProc-users mailing list
> BPr...@li...
> https://lists.sourceforge.net/lists/listinfo/bproc-users
>

On Tue, Dec 18, 2001 at 10:55:39AM +0100, Carlos J. Garcia Orellana wrote:
> One question more: Is necesary to have libbeostat and libbeomap to use mpi
> with Erik's patch?

No.  The patch doesn't address scheduling at all.  It's up to mpirun
to chose nodes and place the right number of jobs on each node.
mpirun can use anything at all to make those decisions.

The mpirun I wrote (which I will probably get out some time after
xmas) doesn't do any resource allocation at all.  We were hoping to
trick somebody err... I mean encourage somebody to write an open
scheduler for a system like this.  We'll probably try to allocate a
student to that probalem some time in the future.

> Last weekend, I've modificated libbeostat to run with 2.4.13 kernel and now
> I have working a mpi 1.2.2 build as rpm package (using the spec file
> provided by Scyld in utils dir of mpich 1.2.2 distribution). I looks work
> fine.
> 
> Are there any problem in using libbeostat?.

Nope.  Use whatever you want.

- Erik
-- 
Erik Arjan Hendriks          Printed On 100 Percent Recycled Electrons
er...@he...                   Contents may settle during shipment