OpenSSI Clusters for Linux / News: Recent posts

OpenSSI leverages HP's NonStop Clusters for Unixware and various open source technologies to provide a full, highly-available SSI (Single System Image) clustering environment for Linux. Goals include availability, scalability and manageability. The source code has been completely reorganized. There are new instructions for installation. This release features improvements to the OpenSSI-enhanced /dev filesystem and LVS. Furthermore, it can now migrate processes linked to libpthread, including Perl processes. See the release notes for more details.

Since 0.8.0, OpenSSI has been booted on more than 67 IA-32 nodes. It includes a first cut of HA-CFS, which allows the cluster to lose its CFS root filesystem server node and continue running, if another node is attached to the same disk. The Lustre and NFS clients were integrated (non-root). LVS now automatically registers any socket that does a listen(). Unix domain sockets, SysV shared memory, and SysV semaphores are all clusterwide.... read more

OpenSSI has been tested upto 27 IA-64 nodes and 41 IA-32 nodes. It includes a first cut of being able to failover a hard mounted CFS root filesystem. /dev is now a context symlink into devfs. Semaphores and Unix domain datagram sockets are now clusterwide. The Lustre client has been integrated for non-root filesystems. A /proc interface is now used to migrate a process, rather than sending it a special signal. Various bugs have been fixed, including problems with strace on IA64 and Alpha.

This release includes a first cut of HA-CFS for non-root file systems. See README.hardmounts for details. The bind() system call now automatically registers a socket with IPVS. Build IPVS static (not as a module) to take advantage of this feature. The permissions were changed for the /proc/cluster/nodeX files so that everyone can read them. The ypbind daemon now works correctly thanks to a CFS fix. Various process signalling fixes were done, along with other miscellaneous bug fixes. Includes Cluster Tools 0.8.0.

Cluster signal numbers have been moved to resolve a conflict with libpthread. Cluster system call numbers have been moved to avoid numbers claimed by the Red Hat kernel. This release fixes a loadlevel inheritance bug and a vproc refcnt bug. It also prevents a panic that occurred when a process did a SOL_SOCKET setsockopt() or getsockopt() on a remote socket. Due to these changes, you must upgrade to Cluster Tools 0.7.6 to use this release.

New is a binary release of OpenSSI for Red Hat 7.3. It includes a modified 2.4.18-18.7.x kernel RPM, enhanced with support for SSI clustering. It also includes a Cluster Tools RPM, containing utilities that support the configuration, operation, and management of an SSI cluster. Read INSTALL in the RPM tarball for instructions on installing and configuring this SSI clustering software.

Merged with Linux 2.4.18. Added rexec loadleveling so that overloaded
nodes will do new execs on underloaded nodes. Distributed certain files
in /proc/cluster. Improved responsiveness of load leveler. Added support
for /proc/<pid>/goto interface. Added some new features to the loadlevel
command. Added code for Context-Dependent Symlinks (CDSLs). Implemented
support for gracefully handling the loss of a non-root CFS server.
Enhanced mount, swapon and fsck to support a clusterwide fstab. Updated
the IPVS and libipvs source. Cleaned up the documentation a bit.

This project won the Best Open Source Project award of the 2002 LinuxWorld Conference and Expo. For pictures and a copy of the demo programs we used, visit http://ssic-linux.sf.net/.

The biggest feature enhancement is coherent cluster-wide shared mapped
file support through CFS. This allows programs such as RPM and keepalive
to work with a CFS root. Another enhancement is the addition of a few
more remote socket ops that allow accept(), getsockname(), and
getpeername() to be called on a socket that is remote from its process.
Also, --bind mounts are now completely local, which is a quick-and-dirty
way to allow context-dependent symlinks (CDSLs). True CDSLs will be
available sometime in the future.... read more

The Cluster File System can be used for the root, removing the need for
expensive shared disk hardware!! Architectures supported now include IA-64.
Up to 32 nodes in a cluster have been tested, and up to 125 may work.
Load-leveling has been enhanced. Mounts are distributed clusterwide.
Some socket ops have been made to work remote. Various bugs have been
fixed.

Highlights include working support for UML (including migrate and rfork), clusterized /proc/<pid>, support for remote ptrace(), and a working version of CFS with some limitations.

A preliminary draft of the Creating SSI Clusters Using UML HOWTO is now available at:
http://ssic-linux.sf.net/ssi-uml-howto/t1.htm

Feedback is most certainly welcome!

The CVS repository is now available. I added some CVS links on the project homepage. I also set up a ssic-linux-checkins mailing lists for receiving automatic checkin notifications. If you subscribe to ssic-linux-checkins, you may also want to subscribe to ci-linux-checkins, since SSI is based on CI.

To build a SSI kernel from a CVS sandbox, read both INSTALL and INSTALL.cvs in the root of the sandbox.

The SSI project leverages both Compaq's NonStop Clusters for Unixware technology and other open source technology to provide a full, highly available SSI environment for Linux. Goals for SSI Clusters include availability, scalability and manageability.

Release notes:
- Based on 2.4.16 (John Byrne)
- Integrated with OpenGFS 0.0.92 (plus some bug fixes) (Dave Zafman)
- Greatly improved installation and booting of non-initial nodes. Uses netboot for non-initial-node installation, rather than requiring a distribution to be installed on every node. Netboot may be optionally used all the time for booting any given node. (Brian Watson)
- Process load leveling, using a variant of the Mosix load leveling algorithm (see README.mosixll) (Laura Ramirez)
- Support for Alpha processor (Aneesh Kumar / John Byrne)
- Preliminary UML support (instructions will come out later) (Kitrick Sheets)
- New version of CI (Cluster Infrastructure 0.6) with a better node detection mechanism and a cleaner way to integrate other kernel subsystems (Kai-Min Sung)
- Better way to set up networking on the non-initial node (previously only the interconnect networking was up on those node) (Kai-Min Sung)
- Initial version of LVS integration (Kai-Min Sung)
- Initial checkin of some CFS (Cluster Filesystem) code; this isn't buildable yet (Dave Zafman)
- Support for Debian (Aneesh Kumar)

Bruce wrote 'Introduction to Single System Image Clustering' for the SourceForge Cluster Foundry:
http://foundries.sourceforge.net/clusters/index.pl?node_id=38692

A flattering article in The Register on Nov 14:
http://www.theregister.co.uk/content/53/22849.html

...during an interview with Newsforge on Nov 8:

"I really want to see the Compaq clustering code, the IBM DLM and OpenGFS in the 2.5 tree creating a real clustered Linux with true failover facilities. That will really open the door to the enterprise market."

To read interview:
http://www.newsforge.com/article.pl?sid=01/11/07/1516223&mode=thread

The SSI project leverages both Compaq's NonStop Clusters for Unixware technology and other open source technology to provide a full, highly available SSI environment for Linux.

This latest release of SSI Linux is based on Alan Cox's 2.4.10-ac4 kernel tree and includes the latest version of OpenGFS. Also integrated in the patch file is SGI's kernel debugger kdb-v1.9. See the CHANGES log for more details.

Cluster Infrastructure (CI) for Linux, a major component of SSI Clusters for Linux, has been released. It includes our kernel-to-kernel communication subsystem (ICS) and cluster membership service (CLMS). CI can also be used as the foundation for more loosely coupled clusters. Please visit the CI project summary page at http://sourceforge.net/projects/ci-linux/.

The full SSI code will be released Real Soon Now (TM)...