Menu
▾
▴
bproc-users — General discussion about BProc.
You can subscribe to this list here.
| 2001 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(25) |
Nov
|
Dec
(22) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2002 |
Jan
(13) |
Feb
(22) |
Mar
(39) |
Apr
(10) |
May
(26) |
Jun
(23) |
Jul
(38) |
Aug
(20) |
Sep
(27) |
Oct
(76) |
Nov
(32) |
Dec
(11) |
| 2003 |
Jan
(8) |
Feb
(23) |
Mar
(12) |
Apr
(39) |
May
(1) |
Jun
(48) |
Jul
(35) |
Aug
(15) |
Sep
(60) |
Oct
(27) |
Nov
(9) |
Dec
(32) |
| 2004 |
Jan
(8) |
Feb
(16) |
Mar
(40) |
Apr
(25) |
May
(12) |
Jun
(33) |
Jul
(49) |
Aug
(39) |
Sep
(26) |
Oct
(47) |
Nov
(26) |
Dec
(36) |
| 2005 |
Jan
(29) |
Feb
(15) |
Mar
(22) |
Apr
(1) |
May
(8) |
Jun
(32) |
Jul
(11) |
Aug
(17) |
Sep
(9) |
Oct
(7) |
Nov
(15) |
Dec
|
|
From: Wilton W. <ww...@ha...> - 2002-04-24 07:46:37
|
This patch lets you use modversions properly with the bproc module, it will generate the proper ksyms in the EXPORT_SYMBOL's. Hopefully I have not broken "non-modversion" compiles, I haven't tested this. - Wilton ----[ Wilton William Wong ]--------------------------------------------- 11060-166 Avenue Ph : 01-780-456-9771 High Performance UNIX Edmonton, Alberta FAX: 01-780-456-9772 and Linux Solutions T5X 1Y3, Canada URL: http://www.harddata.com -------------------------------------------------------[ Hard Data Ltd. ]---- |
|
From: Wilton W. <ww...@ha...> - 2002-04-24 07:44:56
|
These are patches against beoboot-lanl.1.2 the bigkernel patch corrects a bug where bzImaged kernels that are "big" cannot be loaded by kmonte, the second allows you to use modutils-2.4.14 - Wilton ----[ Wilton William Wong ]--------------------------------------------- 11060-166 Avenue Ph : 01-780-456-9771 High Performance UNIX Edmonton, Alberta FAX: 01-780-456-9772 and Linux Solutions T5X 1Y3, Canada URL: http://www.harddata.com -------------------------------------------------------[ Hard Data Ltd. ]---- |
|
From: Erik A. H. <er...@he...> - 2002-04-04 17:09:41
|
On Wed, Apr 03, 2002 at 03:30:11PM -0500, he...@se... wrote: > I noticed the vmadump disclaimer about only being able to dump one thread. > This seems to be the problem with running java right? -- Is there any hope > for the poor misguided souls on our clusters who want to run Java. Well, you can't migrate anything multi-threaded. That's kernel level threads, btw. User level threads all live in a single process and those should be ok. If a user wants to migrate something to a node and then start multiple threads once it's there, that *should* be ok. Multi-thread support in VMADump is on the to-do list somewhere but I don't see myself getting to it any time soon. It's a non-trivial modification. - Erik -- Erik Arjan Hendriks Printed On 100 Percent Recycled Electrons er...@he... Contents may settle during shipment |
|
From: Erik A. H. <er...@he...> - 2002-04-04 17:05:23
|
On Wed, Apr 03, 2002 at 11:47:20AM -0500, Jag wrote: > I just noticed a bug in the /usr/include/sys/bproc.h file for BProc > 3.1.9 Apparently the _bproc_vrfork() function became > _bproc_vrfork_io(), however the header file was never adjusted to > represent this. So, the header files mentioned a _bproc_vrfork() which > doesn't exist, and doesn't mention the actual function, > _bproc_vrfork_io() Ah, thanks. It'll be fixed in the next rev along with a "vexecmove" version of the call. Note that vrfork in 3.1.9 is still a little half baked. - Erik |
|
From: <he...@se...> - 2002-04-03 22:19:44
|
I noticed the vmadump disclaimer about only being able to dump one thread. This seems to be the problem with running java right? -- Is there any hope for the poor misguided souls on our clusters who want to run Java. Nic Nicholas Henke Undergraduate - Engineering 2002 -- Senior Architect and Developer Liniac Project - University of Pennsylvania http://clubmask.sourceforge.net |
|
From: Jag <ag...@li...> - 2002-04-03 16:48:20
|
I just noticed a bug in the /usr/include/sys/bproc.h file for BProc 3.1.9 Apparently the _bproc_vrfork() function became _bproc_vrfork_io(), however the header file was never adjusted to represent this. So, the header files mentioned a _bproc_vrfork() which doesn't exist, and doesn't mention the actual function, _bproc_vrfork_io() Sean |
|
From: Erik G. <er...@gr...> - 2002-03-28 05:24:30
|
> I upgraded to new version of Clustermatic: > > http://www.clustermatic.org > > and everything worked smoothly. Same here. The upgrade went smoothly, after running depmod as suggested the nodes came up, and using the included mpich and mpirun sources, mpi jobs now run on the nodes. Nifty indeed. Erik > > (Clustermatic is BProc-based like Scyld distribution but uses latest > BProc and builds on RedHat 7.2. Clustermatic distro is much smaller > than Scyld distro but for me it contains all I need - slaves boot from > master, bpsh migrates jobs and 'ps -auxf' shows whole cluster.) > > Upgrade is easy, just few notes to complement README: > > - there are iso CD images but rpm upgrade from previous Clustermatic > (or directly from RedHat7.2) is IMHO even easier: > > 1) Go to: > http://www.clustermatic.org/download/CMmarch2002/RPMS/ > > 2) Get specific rpms (e.g. i686) where they exist (kernel, modules) > (ignore zero length files if you see any) > > 3) Get less specific rest from i386 > > Select either SMP or non-SMP versions and simply put all rpm files to > one directory. There is not many of them, it is easy to select by > hand. > > 4) Upgrade your system according to README > > There are just few minor things to watch out during step 4): > > - if you want boot floppy, do this BEFORE you do rpm -Uvh: > > modprobe vfat > modprobe loop > modprobe msdos > > to make sure you have these old modules while you need them last time. > > - rpm -Uvh may warn you about 'config' and 'fstab' config files - you > probably want to merge old and new ones, at least you want to edit > kernel name in 'config'. > > - there is slight ommision in README, command to create phase1 images > is: > > beoboot -1 -k /boot/vmlinuz-2.4.18-lanl.16beoboot -f -o /dev/fd0 > > - if you use some old Scyld phase1 images, you will need to upgrade > them > > - you will probably want to run 'depmod', as README suggests > > For some of you, this might be all you need and want on your cluster - > just add opensource batch spooling: > > http://gridengine.sunsource.net/project/gridengine/download.html > > GridEngine beta2 can most likely work with things above just using bpsh > in simple StarterMethod script (using one queue per node, for > non-parallel jobs), I'll be back with details once I finish this... > > > Best Regards > > Vaclav Hanzl > > _______________________________________________ > BProc-users mailing list > BPr...@li... > https://lists.sourceforge.net/lists/listinfo/bproc-users -- Erik Green er...@gr... |
|
From: Rayson Ho <ray...@ya...> - 2002-03-28 04:20:21
|
Just curious, how well do SGE+Clustermatic scale?? I remember someone on this list asked for tips about running a batch system on a beowulf, can you also tell us the experience (and cluster setup) in your next email?? Also, any advantage using a batch system w/ beowulf?? Thanks, Rayson > For some of you, this might be all you need and want on your cluster > - > just add opensource batch spooling: > > http://gridengine.sunsource.net/project/gridengine/download.html > > GridEngine beta2 can most likely work with things above just using > bpsh in simple StarterMethod script (using one queue per node, for > non-parallel jobs), I'll be back with details once I finish this... __________________________________________________ Do You Yahoo!? Yahoo! Movies - coverage of the 74th Academy Awards® http://movies.yahoo.com/ |
|
From: <ha...@no...> - 2002-03-27 16:42:18
|
I upgraded to new version of Clustermatic: http://www.clustermatic.org and everything worked smoothly. (Clustermatic is BProc-based like Scyld distribution but uses latest BProc and builds on RedHat 7.2. Clustermatic distro is much smaller than Scyld distro but for me it contains all I need - slaves boot from master, bpsh migrates jobs and 'ps -auxf' shows whole cluster.) Upgrade is easy, just few notes to complement README: - there are iso CD images but rpm upgrade from previous Clustermatic (or directly from RedHat7.2) is IMHO even easier: 1) Go to: http://www.clustermatic.org/download/CMmarch2002/RPMS/ 2) Get specific rpms (e.g. i686) where they exist (kernel, modules) (ignore zero length files if you see any) 3) Get less specific rest from i386 Select either SMP or non-SMP versions and simply put all rpm files to one directory. There is not many of them, it is easy to select by hand. 4) Upgrade your system according to README There are just few minor things to watch out during step 4): - if you want boot floppy, do this BEFORE you do rpm -Uvh: modprobe vfat modprobe loop modprobe msdos to make sure you have these old modules while you need them last time. - rpm -Uvh may warn you about 'config' and 'fstab' config files - you probably want to merge old and new ones, at least you want to edit kernel name in 'config'. - there is slight ommision in README, command to create phase1 images is: beoboot -1 -k /boot/vmlinuz-2.4.18-lanl.16beoboot -f -o /dev/fd0 - if you use some old Scyld phase1 images, you will need to upgrade them - you will probably want to run 'depmod', as README suggests For some of you, this might be all you need and want on your cluster - just add opensource batch spooling: http://gridengine.sunsource.net/project/gridengine/download.html GridEngine beta2 can most likely work with things above just using bpsh in simple StarterMethod script (using one queue per node, for non-parallel jobs), I'll be back with details once I finish this... Best Regards Vaclav Hanzl |
|
From: Erik A. H. <er...@he...> - 2002-03-26 21:53:55
|
On Tue, Mar 26, 2002 at 08:50:55PM +0100, ha...@no... wrote: > (By the way, I noticed lots of zero length files at > > http://www.clustermatic.org/download/CMmarch2002/RPMS/i686/ > > and also at athlon dir. Most likely one can just safely ignore them but > you might want to delete them on the web anyway.) Hrm... Thanks for pointing that out. There are *supposed* to be symlinks pointing to the i386 RPM for the RPMs where there is no i686 or athlon specific RPM. I'll see what I can do about getting that fixed on the web site. - Erik -- Erik Arjan Hendriks Printed On 100 Percent Recycled Electrons er...@he... Contents may settle during shipment |
|
From: <ha...@no...> - 2002-03-26 19:40:54
|
> A new version of the clustermatic CD was placed on > www.clustermatic.org today. Great, thanks. Just downloading... I suppose upgrade will be smooth ;-) (By the way, I noticed lots of zero length files at http://www.clustermatic.org/download/CMmarch2002/RPMS/i686/ and also at athlon dir. Most likely one can just safely ignore them but you might want to delete them on the web anyway.) Regards Vaclav |
|
From: Erik G. <er...@gr...> - 2002-03-26 04:48:13
|
Boy, do I have great timing or what? Seriously, thanks Erik. /me goes off to download. Erik Green Erik Arjan Hendriks said: > A new version of the clustermatic CD was placed on > www.clustermatic.org today. The diff looks like this: [deletions] > The MPICH hacks which I've mentioned are included on the CD in the > mpirun-0.1 tar ball in the tarballs directory. We used those hacks to > run a 396 process job on Chiba city at Argonne National Lab the other > day. This clustermatic also survived some boot time torture testing on > about 200 nodes there. (Thanks guys!) The load spike on the front end > is severe when they all boot at once but all the nodes we started with > came up. We've got a guy working on that problem here. > Hopefully beoboot lanl 1.3 will mostly eliminate that problem. > -- Erik Green er...@gr... |
|
From: Erik A. H. <er...@he...> - 2002-03-26 03:53:15
|
A new version of the clustermatic CD was placed on www.clustermatic.org today. The diff looks like this: * Updated BProc (3.1.9): fewer bugs, remote exec hacks, etc. * Updated Beoboot (lanl 1.2): minor changes. * PPC support added. There's no two kernel monte on ppc right now but everything else is there. The ISO9660 filesystem image is bootable on all three architectures. On intel and alpha, it comes up with a phase 1 boot image and on ppc it comes up with phase 2. The MPICH hacks which I've mentioned are included on the CD in the mpirun-0.1 tar ball in the tarballs directory. We used those hacks to run a 396 process job on Chiba city at Argonne National Lab the other day. This clustermatic also survived some boot time torture testing on about 200 nodes there. (Thanks guys!) The load spike on the front end is severe when they all boot at once but all the nodes we started with came up. We've got a guy working on that problem here. Hopefully beoboot lanl 1.3 will mostly eliminate that problem. - Erik -- Erik Arjan Hendriks Printed On 100 Percent Recycled Electrons er...@he... Contents may settle during shipment |
|
From: Erik G. <er...@gr...> - 2002-03-25 21:18:22
|
I thought I'd mention to the group - I did an install of Clustermatic over Red Hat 7.2, and things went well, up to the point where I tried to make MPICH work. Clustermatic is very nice and simple to install, by the way. I understand from the list archive that the patches to make the latest bproc work with MPICH aren't out yet... is there any software someone could recommend to bang on my new install? I can locate some MPI based code, but I can't seem to find much written to just use bproc to distribute work. I want to make sure my setup works properly before I make it non functional with my own code =) Erik Green PS: My lab cluster is a surplus shop special - 4x Ppro 180, 64M each, 100mbit switch, linked to a 1.4Ghz athlon on the front end. Total cost not counting the athlon is about USD 260. -- Erik Green er...@gr... |
|
From: Erik A. H. <er...@he...> - 2002-03-25 14:51:12
|
> If it appears and is good and installed anyway, it may be nice to > synchronize its file caching activities with BProc library caching to > avoid any duplicity? (But probably this is too hypothetical today.) I would expect the library caching stuff to simply disappear if an appropriate caching file system comes along. - Erik -- Erik Arjan Hendriks Printed On 100 Percent Recycled Electrons er...@he... Contents may settle during shipment |
|
From: <ha...@no...> - 2002-03-25 10:38:03
|
> > (Just "cat kernel>/dev/fd0; rdev ..." will not help if your root > > partition is RH72 default ext3.) > ... > The ext2 filesystem code should, however, be able to mount an ext3 > file system. You just won't get the journalling features. Kernel said that it cannot mount root fs because of unsupported options. (Maybe there is a way to force it, I did not try.) > > My question is: How long should I expect my phase1 images to stay? How > > long will be my "forever" this time? > > Well, I'm not going to say "forever" but I would expect them to be ok > for at least the next few revs. > ... > Having to come up with a working image and doing the re-flashing step > on those machines always makes me really nervous. > > I try to avoid changing the phase 1 boot as much as possible. OK, thanks, that's quite good guarantee for me :-) Regards Vaclav |
|
From: <ha...@no...> - 2002-03-25 10:25:33
|
> > Given this situation, what are the future strategic directions for > > BProc to go? > > > > I can imagine this: Quite general file cache in RAM and local > > harddisks, not only for libraries but also for executables and > > data. This cache may or may not be part of BProc but should > > collaborate closely. > > > > ... > > What you've described here is a network file system with caching > ablilities. I think that problem is independent from BProc. I have > no plans to try and solve it well within BProc. The library caching > stuff that's a part of BProc right now is basically a really crude > network file system with no support for coherency at all. OK. Well defined and well limited things are good. I am not aware of any good opensource network file system with local hd caching but quite likely it will appear one day (I hope). If it appears and is good and installed anyway, it may be nice to synchronize its file caching activities with BProc library caching to avoid any duplicity? (But probably this is too hypothetical today.) Regards Vaclav |
|
From: Erik A. H. <er...@he...> - 2002-03-25 02:05:29
|
On Sun, Mar 24, 2002 at 01:08:18PM +0100, ha...@no... wrote: > I installed Clustermatic according to included README (on top of fresh > RH7.2). Most things worked great, I have just these little notes: Cool. > 1) After "rpm -Uvh *" previous kernel modules are gone. This is a > problem if you still need them to make system bootable with new > kernel. I used floppy boot (did not want to touch some things on hd > during these experiments) and was unable to create new boot floppy > without old fat.o msdos.o vfat.o and loop.o. > > If you want to avoid this trap, copy /lib/modules somewhere before > kernel upgrade. > > (Just "cat kernel>/dev/fd0; rdev ..." will not help if your root > partition is RH72 default ext3.) Yeah, I've noticed that myself. That's an artifact of the way the Red Hat RPMs work, I'm afraid. I imagine you should be able to install "kernel" and "kernel-smp" with -i instead of -U. They shouldn't conflict with the existing kernel packages. The ext2 filesystem code should, however, be able to mount an ext3 file system. You just won't get the journalling features. > 2) I was rather disappointed that my old Scyld phase1 images on nodes > did not work with Clustermatic. It is likely that I should blame the > old system and not the new one, but anyway - I somehow believed that > once the node ever beoboots, I never ever have to touch phase1 images > again (unless I change the hardware). If it RARPs and starts once, it > should RARP and start with future master node setups as well. > > I was wrong. My old Scyld beoboot 1.1.16 with 3c905B and driver > v0.99Rb 8/8/2000 just RARPs forever with Clustermatic master (while it > works with old Scyld master). > > OK, I will just upgrade phase1 images on all local harddisks... :-( > > My question is: How long should I expect my phase1 images to stay? How > long will be my "forever" this time? Well, I'm not going to say "forever" but I would expect them to be ok for at least the next few revs. Basically the following changed since scyld: * Two Kernel Monte boot image changed a bit to accomodate Alpha boot images. * The RARP step was extended to include an architecture tag and in the case phase2 BProc version stuff. * Boot images are downloaded with the same multicast protocol as the libraries. The next thing that I can see which would require a change in phase 1 boot images would be an improved multicast protocol for library and boot image distribution. The current protocol is just something I made up. It seems to mostly work but I know it sucks. I don't have any replacement in mind right now. I just know the current implementation kinda sucks. I do understand your annoyance with changing the phase 1 images. On our cluster the phase 1 image is stored in flash along with LinuxBIOS. Having to come up with a working image and doing the re-flashing step on those machines always makes me really nervous. I try to avoid changing the phase 1 boot as much as possible. - Erik |
|
From: Erik A. H. <er...@he...> - 2002-03-25 01:48:25
|
On Wed, Mar 20, 2002 at 11:54:40AM +0100, ha...@no... wrote: > > > Moving the user's script as a whole is still tempting... > > > > That is what we tend to do for the embarrasingly parallel batch jobs, the > > wrapper ( master node bpsh... script) just executes a ton of the indivual > > scripts. What we have is the binaries mounted on GigE NFS/PVFS so that we > > dont have to move the binaries over bproc, just the script that will call > > the programs. It seems to work pretty well. > > Ah, here we are finally. One could think that dirs like /bin and > /usr/bin network-mounted on nodes are insane (and running scripts on nodes > is insane) once you have BProc but real-life statistics say otherwise: > > - it happens in Clubmask, as indicated above > - it happens in my system, I did not know otherwise > - commercial Scyld port of PBS is likely to do it cause probably there > is no other way to run standard PBS user script jobs (even more > likely given the old version of BProc they use) > - Clustermatic does not do it (?) (yet? - no batch spooling there yet) We have our own effort to create a simple and hopefully elegant scheduler specifically for BProc type environments. The guy writing tells me it's pretty much ready to test on some realy boxes so I'm going to be taking a closer look at it this week. The internals are based on the Distributed Job Manager (DJM) from the CM-5. No attempt has been made to provide an interface that's compatible with anything out there right now. It does not directly start anything on BProc nodes. It simply does bproc_chown, etc. on nodes and then runs a script on the front end. An environment variable contains a list of node numbers that mpirun or whatever should run processes on. I'm trying to get people out of the habit of running some script on every node. > On the other side, nice unified PID space still works nicely with > network-mounted binaries as long as all processes which are to belong > to unified PID space are descendants of processes BProc-moved to > nodes, right? Yup. > Given this situation, what are the future strategic directions for > BProc to go? > > I can imagine this: Quite general file cache in RAM and local > harddisks, not only for libraries but also for executables and > data. This cache may or may not be part of BProc but should > collaborate closely. > > This cache could behave as filesystem which mimics whole file > namespace on master and script chrooted to it could be perfectly > happy. > > I would quite enjoy data file cache on slave harddisks. We use huge > sets of files and our script jobs use repeated subsets of them. Local > harddisks are big and much quicker than our network interconnect. (And > I hate copying files around by hand or creating scripts for it; file > cache should take care, not me.) What you've described here is a network file system with caching ablilities. I think that problem is independent from BProc. I have no plans to try and solve it well within BProc. The library caching stuff that's a part of BProc right now is basically a really crude network file system with no support for coherency at all. - Erik |
|
From: <ha...@no...> - 2002-03-24 11:58:37
|
I installed Clustermatic according to included README (on top of fresh RH7.2). Most things worked great, I have just these little notes: 1) After "rpm -Uvh *" previous kernel modules are gone. This is a problem if you still need them to make system bootable with new kernel. I used floppy boot (did not want to touch some things on hd during these experiments) and was unable to create new boot floppy without old fat.o msdos.o vfat.o and loop.o. If you want to avoid this trap, copy /lib/modules somewhere before kernel upgrade. (Just "cat kernel>/dev/fd0; rdev ..." will not help if your root partition is RH72 default ext3.) 2) I was rather disappointed that my old Scyld phase1 images on nodes did not work with Clustermatic. It is likely that I should blame the old system and not the new one, but anyway - I somehow believed that once the node ever beoboots, I never ever have to touch phase1 images again (unless I change the hardware). If it RARPs and starts once, it should RARP and start with future master node setups as well. I was wrong. My old Scyld beoboot 1.1.16 with 3c905B and driver v0.99Rb 8/8/2000 just RARPs forever with Clustermatic master (while it works with old Scyld master). OK, I will just upgrade phase1 images on all local harddisks... :-( My question is: How long should I expect my phase1 images to stay? How long will be my "forever" this time? Anyway, many thanks for Clustermatic which is great. Best Regards Vaclav Hanzl |
|
From: Jag <ag...@li...> - 2002-03-20 21:22:50
|
On Tue, 19 Mar 2002, Erik Arjan Hendriks wrote: > new (and very simple) scheduler for BProc based systems. I believe > Scyld has one too although it's not open source. Scyld has beodiag which is a small library and command line interface to that library that uses libbeostat and a simple algorithm to chose the best X nodes to send jobs to. This is what is used by bbq (at) and mpich on Scyld Beowulf systems to determine where to send nodes to. It /is/ GPLed and can be downloaded from ftp://ftp.scyld.com/pub/beowulf-components (I think its in the beostatus directory off of there) |
|
From: Jag <ag...@li...> - 2002-03-20 13:26:21
|
On Wed, 20 Mar 2002, kilesa wrote: > make kernel ok! > but make bproc error! > #make > make -C vmadump vmadump.o > make[1]: Entering directory `/home/linux/bproc-3.1.9/vmadump' > make[1]: `vmadump.o' is up to date. > make[1]: Leaving directory `/home/linux/bproc-3.1.9/vmadump' > make -C kernel > make[1]: Entering directory `/home/linux/bproc-3.1.9/kernel' > gcc -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -pipe -fno-strength- > reduc > e -D__KERNEL__ -DMODULE -DPACKAGE_VERSION='"3.1.9"' - > DPACKAGE_MAGIC='61127' -DE > NABLE_DEBUG -DLINUX_TCP_IS_BROKEN -I. -I../vmadump - > I/usr/src/linux/include -c g > host.c > ghost.c: In function `ghost_deliver_msg': > ghost.c:74: structure has no member named `bproc' > ghost.c:78: structure has no member named `bproc' > ghost.c: In function `add_ghost': I don't have time to look at the code this minute.. but I'm going to take a stab in the dark and say you're not building against kernel headers that have the BProc patch applied. |
|
From: <ha...@no...> - 2002-03-20 10:45:47
|
> > Moving the user's script as a whole is still tempting... > > That is what we tend to do for the embarrasingly parallel batch jobs, the > wrapper ( master node bpsh... script) just executes a ton of the indivual > scripts. What we have is the binaries mounted on GigE NFS/PVFS so that we > dont have to move the binaries over bproc, just the script that will call > the programs. It seems to work pretty well. Ah, here we are finally. One could think that dirs like /bin and /usr/bin network-mounted on nodes are insane (and running scripts on nodes is insane) once you have BProc but real-life statistics say otherwise: - it happens in Clubmask, as indicated above - it happens in my system, I did not know otherwise - commercial Scyld port of PBS is likely to do it cause probably there is no other way to run standard PBS user script jobs (even more likely given the old version of BProc they use) - Clustermatic does not do it (?) (yet? - no batch spooling there yet) On the other side, nice unified PID space still works nicely with network-mounted binaries as long as all processes which are to belong to unified PID space are descendants of processes BProc-moved to nodes, right? Given this situation, what are the future strategic directions for BProc to go? I can imagine this: Quite general file cache in RAM and local harddisks, not only for libraries but also for executables and data. This cache may or may not be part of BProc but should collaborate closely. This cache could behave as filesystem which mimics whole file namespace on master and script chrooted to it could be perfectly happy. I would quite enjoy data file cache on slave harddisks. We use huge sets of files and our script jobs use repeated subsets of them. Local harddisks are big and much quicker than our network interconnect. (And I hate copying files around by hand or creating scripts for it; file cache should take care, not me.) Best Regards Vaclav |
|
From: kilesa <ki...@fa...> - 2002-03-20 10:28:07
|
make kernel ok! but make bproc error! #make make -C vmadump vmadump.o make[1]: Entering directory `/home/linux/bproc-3.1.9/vmadump' make[1]: `vmadump.o' is up to date. make[1]: Leaving directory `/home/linux/bproc-3.1.9/vmadump' make -C kernel make[1]: Entering directory `/home/linux/bproc-3.1.9/kernel' gcc -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -pipe -fno-strength- reduc e -D__KERNEL__ -DMODULE -DPACKAGE_VERSION='"3.1.9"' - DPACKAGE_MAGIC='61127' -DE NABLE_DEBUG -DLINUX_TCP_IS_BROKEN -I. -I../vmadump - I/usr/src/linux/include -c g host.c ghost.c: In function `ghost_deliver_msg': ghost.c:74: structure has no member named `bproc' ghost.c:78: structure has no member named `bproc' ghost.c: In function `add_ghost': 发件地址:10.75.1.156 贝贝嘹望站邮件系统. |
|
From: Nicholas H. <he...@se...> - 2002-03-19 18:18:39
|
> OK, looked to your docs for the third time and again got more of it... :-) > > There are two types of scripts one could care about: > > 1) Scripts preparing environments for parallel jobs (e.g. using > MPI). Mostly written by cluster administrator, these scripts are > moreorless part of computing system and can contain things like > 'getnodes' and 'bpsh'. Predefined example scripts play this role in > Clubmask. They are part of "Parallel Environment" definitions in Grid > Engine. They should execute on master in BProc-based spooling systems. > > 2) Scripts for non-parallel jobs. Each such script requires one > processor only, does some housekeeping on the entry and exit and most > likely runs few heavy executables (or just one) to do the hard > work. Instead of message passing inside MPI, these jobs read and write > files and they are synchronized using job dependencies (start job 11 > when jobs 1-10 are finished). These jobs are written by users and are > expected to be the same across various implementations of batch > spooling systems. > > > Some sites seem to care about 1) and MPI (or PVM) only. But in some > areas (e.g. our speech recognizer training) problems are best solved > using 2). This is where things start to conflict: > > - I want to use BProc because it makes cluster administration easy > > - I want let my users to install some standard spooling system at > home, at laptops etc., read standard documentation, prepare standard > job scripts, learn and debug > > - I want them to carry unchanged scripts to cluster and just see the > job done much more quickly (well, also finish debuging) > > > Standard spooling systems like GE or PBS expect job scripts to be > executed on slave nodes. BProc does not quite like it. > > The best solution I see so far is to mark heavy executables with > prefix which expands to nothing on laptops and home computers and > expands to 'bpsh right-node' on cluster (where all these scripts would > execute on master, off-loading heavy executables to nodes) > > Probably it is good compromise. But I am not exactly happy to learn > users that they should mark heavy executables and risk master node > overload when they do not. > > Moving the user's script as a whole is still tempting... That is what we tend to do for the embarrasingly parallel batch jobs, the wrapper ( master node bpsh... script) just executes a ton of the indivual scripts. What we have is the binaries mounted on GigE NFS/PVFS so that we dont have to move the binaries over bproc, just the script that will call the programs. It seems to work pretty well. Nic |
1488 messages has been excluded from this view by a project administrator.
