Re: [Xen-devel] Setting up a more complex system scenario

[Ian P. <Ian...@cl...>]

> Questions: 
> 
> 1) For IP addresses, do they need to be sequential?  I have 9
> sequential ones, but the first (DOM0) is not.  I can renumber if
> needed, but it looks like Xen just have separate startup scripts with
> hard-coded IP addresses for each non-DOM0 domain.

You can set them to anything you like. Sequential is just a
convenience provided by the tools.
 
> 2) Can my current /usr (and the rest of the distro) co-exist with Xen,
> or will I be unable to boot a "regular" Linux after all this?

You should be able to co-exist just fine -- all of our systems
boot plain Linux from time to time too.

Various commands often run in the startup scripts e.g. "gpm",
"hwclock" will complain, but the errors are harmless. You can
suppress them by using a different init runlevel for linux and
xenolinux boots (as per the demo CD).

> - I'd like them to share as much of the core OS as is reasonable: /,
> /boot, /etc, /usr, /usr/local etc.  This can be by NFS, or (better)
> mounting ro.  This is for convenience in administration.

I've setup a number of systems with a ro /usr on a raw disk
partition, but with each domain having it's own rw /. The /
partition can either be on a raw partition, virtual disk, or NFS
mount.

If you wish to make the / partition smaller, you can copy the
/bin /sbin and /lib directories into /usr/root and create soft
links in /.  

(You need to be a little careful with this as the startup scripts
need a couple of libraries before executing the mount of /usr --
see README.CD)

> - They must have one or more shared data directories (/globus and
> /shared), again via NFS or ro.  (r+w is not needed).  This is
> for getting applications running.

Either will work.

> - They'll each have their own /data partition, r+w, which will be a
> physically separate drive.  /dev/sdk1, /dev/sdl1, /dev/sdm1, etc.

Virtual disks or raw partitions work well for this.

> - I'm not sure how to handle /home.  It would be nice for a user to
> get his *same* home directory r+w on each system.  NFS r+w?

NFS rw works very well.

> So, things I need to do are:
> 
> 1) Get a correct /usr etc. on hard drive, so I can boot virtual
> domains that will run off of hard drive.  (Currently, they're
> running in RAMfs via CD.)

Yep. I'd do a normal redhat install and then use "cp -a" to move
the data between partitions.

> 2) Give each virtual domain a different /etc/fstab or other
> configuration so that they mount the right drives ro and rw and NFS.

I'd give each domain it' s own / partition. (with links for /bin
/lib and /sbin).

> 3) Code a xenctl startup script for each IP-based domain to insure
> each one is booted properly.  For example, "grid-11" should always go
> to the same IP address, not a different one if, for example, one of
> the other domains fails to start or is accidentally started twice.
> 
> For illustration purposes, grid-11.arsc.edu = 137.229.71.11.
> The main system, peabody.arsc.edu = 137.229.71.6

This should work fine.

We're developing a copy-on-write file system that should make all
of this configuration rather easier in the future...

> I promise to write up all these steps, for others to use, once I get
> things running.  Thanks again for your help, this is truly going 
> to be extremely useful software for my Grid applications development.

Thanks,
Ian