Re: [SSI-devel] Clusterwide unique names for NICs

[Brian J. W. <Bri...@hp...>]

A Bergman wrote:
> I thought the point was kind of make it transparent what node you are 
> running on, so even if a process is migrated it still uses the original 
> device?

The eth interfaces aren't devices. The ifconfig command learns about the 
available interfaces by reading the files in /proc/net and doing various 
ioctls on a dummy socket.

This is not an issue for process migration, since a process' sockets 
stay behind on the old node. Even when we do implement socket migration, 
we can use simpler tricks to make sure the socket can still work 
correctly, such as IP forwarding and aliases.

My main motivation for not making other node's interfaces visible is 
ease of implementation. It both reduces the amount of work needed to be 
done in the kernel and minimizes the changes to the rc scripts. For any 
given node that is booting, the rc scripts only need to configure the 
interfaces that they can see.

This is also good for scaling. If a cluster has hundreds of configured 
interfaces, you don't want any given node to waste too much time on 
interfaces that either are already configured or belong to nodes that 
have not yet come up.

BTW, after writing up this design, I remembered that the ICS interfaces 
(used for the cluster interconnect) need to be treated specially. They 
must be configured before a node joins the cluster, so perhaps they 
should be called eth0 on every node. This shouldn't be a problem, since 
the rc scripts shouldn't bother configuring these interfaces anyway.

> Alternatively, should ifconfig be patched to see all nics and also print 
> out what node they are on?

It's best to avoid patching user-mode stuff. The more we touch, the 
harder it is to install OpenSSI and keep it up-to-date with the latest 
base code. That being said, there's no problem with adding a 
ssi-ifconfig command that runs the base ifconfig command on every node 
and reports the results along with node numbers.

> Also, should this be more a general approach, or is this specific for 
> the NICs. (For example, in the future, should hd/sd names be unique 
> across the cluster?

I think this was Bruce's long-term vision for our SSI-enhanced devfs, 
but for now we're going with the easier to implement solution that we 
released in 0.9.0. It allows a process to see the devices on the node 
where it started (in addition to all the ptys in the cluster) under 
/dev. If it wants to see the devices of other nodes, it has to be 
cluster-aware enough to look under /cluster/nodex/dev. Not perfectly 
SSI, but probably good enough for most purposes.

Brian