From: Jeff Lerman <jclerman@an...> - 2004-07-28 10:22:03
I'm starting to have a lot of questions about CFS. First:
I thought that the following line in /etc/fstab on an SSI system was
supposed to make different physical volumes available depending on the
node on which a process is running:
/dev/hda4 /tmp_test ext2 defaults,node=2:3 0 2
so that (after mounting on nodes #2 and #3) ls /tmp_test on node #2 sees
/dev/2/hda4, and on node #3 the same command sees /dev/3/hda4. In practice
what I find is that the entire cluster sees whatever was mounted most recently
on /tmp_test; the other mounts remain but are hidden.
I get the exact same behavior if I have 2 separate lines, e.g.
/dev/hda4 /tmp_test ext2 defaults,node=2 0 2
/dev/hda4 /tmp_test ext2 defaults,node=3 0 2
I thought, based on README.clustertab on the web site, that the different
nodes should see the different physical disks in the above configuration, but
as noted, they don't. Is that as intended?
Second: Is CFS completely distinct from NFS in terms of transport of data
between nodes? Is there a way of "tuning" CFS similar to NFS rsize, wsize,
etc.? We are concerned since some applications we want to run on our cluster
require local, or at least fast, scratch space, and behave poorly when they
have to use NFS mounts for scratch.
Along the same lines, two loadleveling questions:
1. Is there a way to start a process from a loadleveled shell, and specify
that the process should NOT be loadleveled?
2. Is there a way to enable rexec loadleveling but disable subsequent
migration? I.e., a process starts on the best node but doesn't migrate after
that? That would eliminate worries about scratch space being local when a
process starts, but being non-local when it gets migrated.
Postdoc and sysadmin
Dept. of Molecular and Cell Biology
University of California, Berkeley
From: David B Zafman <david.zafman@hp...> - 2004-07-28 15:55:51
On Jul 28, 2004, at 3:21 AM, Jeff Lerman wrote:
> Second: Is CFS completely distinct from NFS in terms of transport of
> between nodes?
> Is there a way of "tuning" CFS similar to NFS rsize, wsize,
No. CFS works on a page (PAGESIZE) basis.
> We are concerned since some applications we want to run on our cluster
> require local, or at least fast, scratch space, and behave poorly when
> have to use NFS mounts for scratch.
If you want to try creating a scratch filesystem without CFS, you can
use a filesystem type NOT listed in the /cluster/lib/cfstab file.
Be careful, since other nodes won't be able to see the filesystem which
means you can't loadlevel processes that have files open in the scratch
filesystem. You can't migrate/rfork/rexec with files open in the
scratch fs because the target node can't reopen the scratch file.
We have been considering adding a mount option allowing you to mount
filesystems that would get CFS to not use it. This would allow someone
to create a scratch space filesystem with ext2/3, for example.
David B. Zafman | Hewlett-Packard Company
mailto:david.zafman@... | http://www.hp.com
"Computer Science" is no more about computers than astronomy is about
telescopes - E. W. Dijkstra