We have noticed that some programs (particularly monitoring programs to be run
on each node) don't want to see a clusterwide /proc and in fact our current
/proc is clusterwide only really w.r.t. the processes. To allow more
programs to "just work" and give more flexibility of environment (and better
performance in some cases), I would like to propose the following
A process can be running in "clusterwide", "local" or "default" mode. The mode
it is running in will affect what it sees in /proc (and maybe a few other
other places). The mode is inherited and is "default" by default. onnode
would have an option to change it and new commands "clusterwide" and "local"
could be used to launch other programs in the mode desired.
"local" will be a handy mode for clusternode_shutdown.
a. /proc is local only; only pids running locally;
b. kill(-1,...) only signals processes running locally;
c. sysvipc only shows ipc objects locally managed;
sysvipc objects created have a key-space of only the local node
(i.e. another process on another node can create an object with the same
key and they aren't the same object); sysvipc id's will be
clusterwide so there is no confusion w.r.t. which object is being
ipcs run in this mode will show only local ipc objects;
a. /proc/<pid> is all the non-kernel-thread pids in the system;
b. /proc/xxx only exists if it is showing clusterwide information;
currently there is virtually nothing in this category but sysvipc should
This shows a mixed environment where most programs should work.
a. readdir on /proc shows all processes
b. /proc/sysvipc is clusterwide
c. /proc/xxx is local
What to do with ps, top, etc.
top - unmodified, it would probably work the following in the following modes:
local - only show local processes against local cpus, memory, etc.
- pretty much single linux machine semantics
global - probably won't work because some of the /proc files will be missing
we could modify it to leave out some of the info at the top and show
all the processes from all the nodes, indicating which node they are
default - would be like it is today, which shows all processes but the
memory and cpu info at the top is local info.
ps - we will definitely need to modify ps to at least provide some options
on seeing where processes are running or just to see processes on a
particular node. However, with no changes it might work like:
local - single machine, like top above.
From: Brian J. Watson <Brian.J.W<atson@hp...> - 2003-09-16 21:30:23
> a. /proc is local only; only pids running locally;
> b. kill(-1,...) only signals processes running locally;
> c. sysvipc only shows ipc objects locally managed;
> sysvipc objects created have a key-space of only the local node
> (i.e. another process on another node can create an object with the same
> key and they aren't the same object); sysvipc id's will be
> clusterwide so there is no confusion w.r.t. which object is being
> ipcs run in this mode will show only local ipc objects;
Note that this local namespace only applies to sysvipc, which consists
of semaphores, message queues, and shared memory. It does not include
fifos or Unix domain sockets, which are typically named in the
clusterwide file systems. You could manually create a local namespace
for them with context-dependent sysmlinks (CDSLs).