Thread: [SSI-devel] RFC: Major reorg of Cluster Tools
Brought to you by:
brucewalker,
rogertsang
From: Brian J. W. <Bri...@hp...> - 2003-05-21 21:38:12
|
To the CI and OpenSSI communities- The CI and OpenSSI user-mode code has been thrown together in an ad hoc fashion since the beginning of these projects. It has been lumped together into one RPM, source release, and CVS module that supports two projects, two Linux distros, and three architectures. Its structure is inconsistent and confusing to novice users and developers, forbids the replacement of commands owned by base RPMs, and encumbers the production of releases. I propose the following changes to clean-up things: Binary release changes ---------------------- * roughly separate cluster-tools into its CI and OpenSSI components * roll just the CI part as the cluster-tools RPM * spin off the OpenSSI part as an openssi-tools RPM and a set of enhanced base RPMs, such as mount (util-linux), devfsd, nfs-utils, and ipvsadm (ha-lvs) * include kernel-openssi, cluster-tools, openssi-tools, the enhanced base packages, documentation, and a simple installation script in a new OpenSSI binary release * eventually roll a CI binary release containing kernel-ci and cluster-tools * libcluster should retain OpenSSI support in the reduced cluster-tools, since it's not worth the effort to separate this code Source release changes ---------------------- * combine the stripped-down cluster-tools with the CI kernel code to make a new CI source release * continue to include combined CI and OpenSSI code in OpenSSI releases * eliminate Cluster Tools as a separate source release * organize the CI and OpenSSI source trees to reflect the binary packages * organize the large number of commands for openssi-tools into functional groups CVS repository changes ---------------------- * reorganize the repository to match the new source trees * adjust the system of Makefiles to reflect the new source organization * exclusively adopt Aneesh's GNU build system, so that only one set of Makefiles needs to be reorg'd * use a consistent system for supporting multiple distributions and architectures * write scripts for automatically producing source and binary releases This is how I propose laying out the CI repository: ci |-- AUTHORS (was in cluster-tools) |-- COPYING |-- ChangeLog |-- Makefile (rules for rolling releases) |-- Makefile.am (rules for building and installing) |-- NEWS (was in cluster-tools) |-- README |-- cluster-tools | |-- Makefile.am | |-- cmd/ (moved some commands to openssi-tools) | |-- libcluster/ | `-- man/ (moved some pages to openssi-tools) |-- configure.ac (was in cluster-tools) |-- doc/ |-- kernel/ (was ci-linux/ci-kernel) |-- kernel.configs | `-- config.i586 (was in ci-linux/ci-kernel) |-- kernel.patches (was ci-linux/3rd-party) | |-- common/ | `-- i386/ `-- specs |-- cluster-tools.spec (was in binary/rh7.3) `-- kernel.spec This is how I propose laying out the OpenSSI repository: openssi |-- AUTHORS |-- COPYING |-- ChangeLog |-- Makefile (rules for rolling releases) |-- Makefile.am (rules for building and installing) |-- NEWS |-- README |-- configure.ac |-- devfsd/ (was in cluster-tools/ssi) |-- distro-pkgs (distro-specific packages) | |-- debian (?? not sure about Debian pkg names ??) | | |-- init-scripts (was cluster-tools/debian-init-scripts) | | | |-- S01devfsd | | | |-- S10checkroot.sh | | | |-- S35mountall.sh | | | |-- hostname.sh | | | |-- install-deb-initscripts (was in cluster-tools) | | | `-- rcSSI | | `-- specs | | `-- init-scripts.spec | `-- redhat | `-- specs/ |-- doc | |-- INSTALL (was in binary/rh7.3) | |-- INSTALL.cvs | |-- INSTALL.gfs | |-- INSTALL.ia64 | |-- INSTALL.pxe (was in cluster-tools) | |-- INSTALL.source (was ssic-linux/doc/INSTALL) | |-- README-CI | |-- README-mosixll | |-- README.cfs | |-- README.hardmounts | `-- cdsl |-- ipvsadm/ (was cluster-tools/ha-lvs) |-- kernel/ (was ssic-linux/ssi-kernel) |-- kernel.configs | |-- config.alpha (was in ssic-linux/ssi-kernel) | |-- config.i586 (was in ssic-linux/ssi-kernel) | |-- config.ia64 (was in ssic-linux/ssi-kernel) | |-- config.ia642 (was in ssic-linux/ssi-kernel) | |-- config.uml (was in ssic-linux/ssi-kernel) | `-- kernel-2.4.18-i586-ssi.config (was in binary/rh7.3) |-- kernel.patches (was ssic-linux/3rd-party) | |-- common/ | |-- i386/ | `-- ia64/ |-- nfs-utils/ (patched by cluster-tools/ssi/nfslock.patch) |-- openssi-tools | |-- Makefile.am | |-- arch | | |-- Makefile.am | | |-- alpha | | | |-- Makefile.am | | | `-- ssi_arch.pm (was cluster-tools/ssi/ssi_alpha.pm) | | |-- i386 | | | |-- Makefile.am | | | `-- ssi_arch.pm (was cluster-tools/ssi/ssi_i386.pm) | | |-- ia64 | | | |-- Makefile.am | | | `-- ssi_arch.pm (was cluster-tools/ssi/ssi_ia64.pm) | | `-- uml | | | |-- Makefile.am | | |-- clustertab (was in cluster-tools/ssi) | | `-- fstab (was in cluster-tools/ssi) | |-- distro | | |-- Makefile.am | | |-- debian | | | |-- Makefile.am | | | |-- inittab.ssi (was cluster-tools/ssi/inittab.ssi.debian) | | | `-- ssi_distro.pm (was cluster-tools/ssi/ssi_debian.pm) | | `-- redhat | | |-- Makefile.am | | |-- inittab.ssi (was cluster-tools/ssi/inittab.ssi.redhat) | | `-- ssi_distro.pm (was cluster-tools/ssi/ssi_redhat.pm) | |-- fs | | |-- Makefile.am | | |-- cfs_mount.c (was in cluster-tools/cmd) | | |-- cfs_remount.c (was in cluster-tools/cmd) | | |-- cfs_setroot.c (was in cluster-tools/cmd) | | |-- ckroot.ssi (was in cluster-tools/cmd) | | |-- cmount.c (was in cluster-tools/cmd) | | |-- gfscf.cf (was in cluster-tools/ssi) | | |-- mount_remote_root.c (was in cluster-tools/cmd) | | |-- pool0.cf (was in cluster-tools/ssi) | | `-- pool0cidev.cf (was in cluster-tools/ssi) | |-- init.ssi/ (was cluster-tools/ssi/init) | |-- initscripts | | |-- Makefile.am | | |-- rc.modules (was in cluster-tools/ssi) | | |-- rc.nodedown (was in cluster-tools/ssi) | | |-- rc.nodeup (was in cluster-tools/ssi) | | |-- rc.sysinit.nodeup (was in cluster-tools/ssi) | | `-- rc.sysrecover (was in cluster-tools/ssi) | |-- keepalive/ (was in cluster-tools/ssi) | |-- man | | |-- Makefile.am | | |-- keepalive.1M.html (was in cluster-tools/man) | | `-- spawndaemon.1M.html (was in cluster-tools/man) | |-- net | | |-- Makefile.am | | |-- node_hostname.c (was in cluster-tools/cmd) | | `-- setport_weight.c (was in cluster-tools/cmd) | |-- proc | | |-- Makefile.am | | |-- load_level_com.h (was in cluster-tools/cmd) | | |-- loadlevel.c (was in cluster-tools/cmd) | | |-- loads.c (was in cluster-tools/cmd) | | |-- migrate.c (was in cluster-tools/ssi/migrate) | | |-- onall.c (was in cluster-tools/cmd) | | |-- onnode.c (was in cluster-tools/cmd) | | `-- where_pid.c (was in cluster-tools/cmd) | `-- sysadmin | |-- Clustertab.pm (was in cluster-tools/ssi) | |-- Makefile.am | |-- addnode (was in cluster-tools/ssi) | |-- addnode.dev (was in cluster-tools/ssi) | |-- addnode.pm (was in cluster-tools/ssi) | |-- chnode (was in cluster-tools/ssi) | |-- cluster_lilo (was in cluster-tools/ssi) | |-- cluster_mkinitrd (was in cluster-tools/ssi) | |-- mkdhcpd.conf (was in cluster-tools/ssi) | |-- openssi_addnode (was in cluster-tools/ssi) | `-- openssi_cluster_create (was in cluster-tools/ssi) |-- specs | |-- devfsd.spec | |-- ipvsadm.spec | |-- kernel.spec (was binary/rh7.3/kernel-2.4.spec) | |-- nfs-utils.spec | |-- openssi-tools.spec | `-- util-linux.spec `-- util-linux/ (was in cluster-tools/ssi) Let me know what you think, Brian |
From: Aneesh K. K.V <ane...@di...> - 2003-05-22 04:24:13
|
Brian, On Thu, 2003-05-22 at 03:07, Brian J. Watson wrote: [...snip....] You may want to wait for the changes done yesterday by laura to be merged back to the trunk before reorganising the CVS. Also cluster-tools patch from Scott also need to be not applied. It would be better if we wait for a day or so and say no more submits and then reorganise.Then we can get the reorganised RH branch and Trunk built and then open up the CVS for further submit. > > > This is how I propose laying out the CI repository: > > ci > |-- AUTHORS (was in cluster-tools) > |-- COPYING > |-- ChangeLog How are you going to organise the Changelog .I mean what about the previous contents. ? > |-- Makefile (rules for rolling releases) > |-- Makefile.am (rules for building and installing) > |-- NEWS (was in cluster-tools) > |-- README > |-- cluster-tools > | |-- Makefile.am > | |-- cmd/ (moved some commands to openssi-tools) > | |-- libcluster/ > | `-- man/ (moved some pages to openssi-tools) > |-- configure.ac (was in cluster-tools) > |-- doc/ > |-- kernel/ (was ci-linux/ci-kernel) > |-- kernel.configs > | `-- config.i586 (was in ci-linux/ci-kernel) > |-- kernel.patches (was ci-linux/3rd-party) > | |-- common/ > | `-- i386/ > `-- specs > |-- cluster-tools.spec (was in binary/rh7.3) > `-- kernel.spec > > > This is how I propose laying out the OpenSSI repository: > > openssi > |-- AUTHORS > |-- COPYING > |-- ChangeLog > |-- Makefile (rules for rolling releases) > |-- Makefile.am (rules for building and installing) > |-- NEWS > |-- README > |-- configure.ac > |-- devfsd/ (was in cluster-tools/ssi) > |-- distro-pkgs (distro-specific packages) > | |-- debian (?? not sure about Debian pkg names ??) > | | |-- init-scripts (was cluster-tools/debian-init-scripts) > | | | |-- S01devfsd > | | | |-- S10checkroot.sh > | | | |-- S35mountall.sh > | | | |-- hostname.sh > | | | |-- install-deb-initscripts (was in cluster-tools) install-deb-initscripts is not needed this job is now done by openssi_cluster_create. > | | | `-- rcSSI > | | `-- specs > | | `-- init-scripts.spec I guess init-script.spec is going to be script which will help in building debian packages. > | `-- redhat > | `-- specs/ > |-- doc > | |-- INSTALL (was in binary/rh7.3) > | |-- INSTALL.cvs > | |-- INSTALL.gfs > | |-- INSTALL.ia64 > | |-- INSTALL.pxe (was in cluster-tools) > | |-- INSTALL.source (was ssic-linux/doc/INSTALL) > | |-- README-CI > | |-- README-mosixll > | |-- README.cfs > | |-- README.hardmounts > | `-- cdsl > |-- ipvsadm/ (was cluster-tools/ha-lvs) > |-- kernel/ (was ssic-linux/ssi-kernel) > |-- kernel.configs > | |-- config.alpha (was in ssic-linux/ssi-kernel) > | |-- config.i586 (was in ssic-linux/ssi-kernel) > | |-- config.ia64 (was in ssic-linux/ssi-kernel) > | |-- config.ia642 (was in ssic-linux/ssi-kernel) > | |-- config.uml (was in ssic-linux/ssi-kernel) > | `-- kernel-2.4.18-i586-ssi.config (was in binary/rh7.3) > |-- kernel.patches (was ssic-linux/3rd-party) > | |-- common/ > | |-- i386/ > | `-- ia64/ > |-- nfs-utils/ (patched by cluster-tools/ssi/nfslock.patch) > |-- openssi-tools > | |-- Makefile.am > | |-- arch > | | |-- Makefile.am > | | |-- alpha > | | | |-- Makefile.am > | | | `-- ssi_arch.pm (was cluster-tools/ssi/ssi_alpha.pm) > | | |-- i386 > | | | |-- Makefile.am > | | | `-- ssi_arch.pm (was cluster-tools/ssi/ssi_i386.pm) > | | |-- ia64 > | | | |-- Makefile.am > | | | `-- ssi_arch.pm (was cluster-tools/ssi/ssi_ia64.pm) > | | `-- uml > | | | |-- Makefile.am > | | |-- clustertab (was in cluster-tools/ssi) > | | `-- fstab (was in cluster-tools/ssi) > | |-- distro > | | |-- Makefile.am > | | |-- debian > | | | |-- Makefile.am > | | | |-- inittab.ssi (was cluster-tools/ssi/inittab.ssi.debian) > | | | `-- ssi_distro.pm (was cluster-tools/ssi/ssi_debian.pm) > | | `-- redhat > | | |-- Makefile.am > | | |-- inittab.ssi (was cluster-tools/ssi/inittab.ssi.redhat) > | | `-- ssi_distro.pm (was cluster-tools/ssi/ssi_redhat.pm) > | |-- fs > | | |-- Makefile.am > | | |-- cfs_mount.c (was in cluster-tools/cmd) > | | |-- cfs_remount.c (was in cluster-tools/cmd) > | | |-- cfs_setroot.c (was in cluster-tools/cmd) > | | |-- ckroot.ssi (was in cluster-tools/cmd) > | | |-- cmount.c (was in cluster-tools/cmd) > | | |-- gfscf.cf (was in cluster-tools/ssi) > | | |-- mount_remote_root.c (was in cluster-tools/cmd) > | | |-- pool0.cf (was in cluster-tools/ssi) > | | `-- pool0cidev.cf (was in cluster-tools/ssi) > | |-- init.ssi/ (was cluster-tools/ssi/init) > | |-- initscripts > | | |-- Makefile.am > | | |-- rc.modules (was in cluster-tools/ssi) > | | |-- rc.nodedown (was in cluster-tools/ssi) > | | |-- rc.nodeup (was in cluster-tools/ssi) > | | |-- rc.sysinit.nodeup (was in cluster-tools/ssi) > | | `-- rc.sysrecover (was in cluster-tools/ssi) > | |-- keepalive/ (was in cluster-tools/ssi) > | |-- man > | | |-- Makefile.am > | | |-- keepalive.1M.html (was in cluster-tools/man) > | | `-- spawndaemon.1M.html (was in cluster-tools/man) > | |-- net > | | |-- Makefile.am > | | |-- node_hostname.c (was in cluster-tools/cmd) > | | `-- setport_weight.c (was in cluster-tools/cmd) > | |-- proc > | | |-- Makefile.am > | | |-- load_level_com.h (was in cluster-tools/cmd) > | | |-- loadlevel.c (was in cluster-tools/cmd) > | | |-- loads.c (was in cluster-tools/cmd) > | | |-- migrate.c (was in cluster-tools/ssi/migrate) > | | |-- onall.c (was in cluster-tools/cmd) > | | |-- onnode.c (was in cluster-tools/cmd) > | | `-- where_pid.c (was in cluster-tools/cmd) > | `-- sysadmin > | |-- Clustertab.pm (was in cluster-tools/ssi) > | |-- Makefile.am > | |-- addnode (was in cluster-tools/ssi) > | |-- addnode.dev (was in cluster-tools/ssi) > | Can't we drop addnode and addnode.dev and use openssi_cluster_create and openssi_addnode > |-- addnode.pm (was in cluster-tools/ssi) > | |-- chnode (was in cluster-tools/ssi) > | |-- cluster_lilo (was in cluster-tools/ssi) > | |-- cluster_mkinitrd (was in cluster-tools/ssi) > | |-- mkdhcpd.conf (was in cluster-tools/ssi) > | |-- openssi_addnode (was in cluster-tools/ssi) > | `-- openssi_cluster_create (was in cluster-tools/ssi) > |-- specs > | |-- devfsd.spec > | |-- ipvsadm.spec > | |-- kernel.spec (was binary/rh7.3/kernel-2.4.spec) > | |-- nfs-utils.spec > | |-- openssi-tools.spec > | `-- util-linux.spec > `-- util-linux/ (was in cluster-tools/ssi) > > > Let me know what you think, Great!!!!. -aneesh |
From: Aneesh K. K.V <ane...@di...> - 2003-05-22 05:06:36
|
On Thu, 2003-05-22 at 10:07, Aneesh Kumar K.V wrote: > You may want to wait for the changes done yesterday by laura to be > merged back to the trunk before reorganising the CVS. Also cluster-tools > patch from Scott also need to be not applied. That should be read as patch from Scott NEED to be Applied. -aneesh |
From: Brian J. W. <Bri...@hp...> - 2003-05-23 05:19:05
|
Aneesh Kumar K.V wrote: > It would be better if we wait for a day or so and say no more submits > and then reorganise.Then we can get the reorganised RH branch and Trunk > built and then open up the CVS for further submit. That's my plan. I'm giving a few days for comments, then I'm going to ask everyone to refrain from checkins while I do the reorg. Then I'll help developers adapt their sandboxes to the new structure. > How are you going to organise the Changelog .I mean what about the > previous contents. ? I'm thinking of ditching the old changes and starting afresh. The history of cluster-tools and ci-linux as separate releases is not that interesting going forward. > install-deb-initscripts is not needed this job is now done by > openssi_cluster_create. Okay. I'll eliminate it. >>| | | `-- rcSSI >>| | `-- specs >>| | `-- init-scripts.spec > > > I guess init-script.spec is going to be script which will help in > building debian packages. In RPM-land, a .spec file provides instructions on how to build the binaries for a package, specifies dependencies on other packages, includes a description of what the package does, etc. Debian might use something different to build its own packages. I'm not sure what. > Can't we drop addnode and addnode.dev and use openssi_cluster_create > and openssi_addnode Not yet. I want to play with them first, make some comments, and see what needs to be changed in the documentation. We can cross this bridge after finishing the reorg. BTW, Scott's change to add PXE support will help me do this. It's been awhile since I've used a test cluster that can do Etherboot, so I haven't even played with the original addnode for a long time. :( >>Let me know what you think, > > Great!!!!. Thanks!! Brian |
From: Aneesh K. K.V <ane...@di...> - 2003-05-23 05:33:35
|
On Fri, 2003-05-23 at 10:48, Brian J. Watson wrote: > Aneesh Kumar K.V wrote: > > It would be better if we wait for a day or so and say no more submits > > and then reorganise.Then we can get the reorganised RH branch and Trunk > > built and then open up the CVS for further submit. > > That's my plan. I'm giving a few days for comments, then I'm going to > ask everyone to refrain from checkins while I do the reorg. Then I'll > help developers adapt their sandboxes to the new structure. > > > > How are you going to organise the Changelog .I mean what about the > > previous contents. ? > > I'm thinking of ditching the old changes and starting afresh. The > history of cluster-tools and ci-linux as separate releases is not that > interesting going forward. > > Then i would request to start with fresh ChangeLogs with new formats for both the repository and old Changelogs can be renamed. ie in the SSI repository we will have ChangeLog and ChangeLog.0( This is the old Changelog ). ChangeLog will have the first line as previous cluster-tools ChangeLog can be found at so and so place in the ci we will have ChangeLog,ChangeLog.0( this is ci-linux change log ) and ChangeLog.cluster-tools( this is cluster-tools changelog ) and we will have a new entry in ChangeLog saying the old cluster-tools ChangeLog can be found in ChangeLog.cluster-tools. I would also say that the ChangeLog fromat can be DD/MM/YYYY username <useremailaddress> * Changes see this . http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/opendlm/opendlm/ChangeLog?rev=HEAD&content-type=text/plain I found this format pretty useful when tracking bugs. > > install-deb-initscripts is not needed this job is now done by > > openssi_cluster_create. -aneesh |
From: Brian J. W. <Bri...@hp...> - 2003-06-03 00:24:49
|
Aneesh Kumar K.V wrote: > Then i would request to start with fresh ChangeLogs with new formats for > both the repository and old Changelogs can be renamed. > > ie in the SSI repository we will have > ChangeLog and ChangeLog.0( This is the old Changelog ). ChangeLog will > have the first line as previous cluster-tools ChangeLog can be found at > so and so place > > > in the ci we will have ChangeLog,ChangeLog.0( this is ci-linux change > log ) and ChangeLog.cluster-tools( this is cluster-tools changelog ) and > we will have a new entry in ChangeLog saying the old cluster-tools > ChangeLog can be found in ChangeLog.cluster-tools. > > I would also say that the ChangeLog fromat can be > > DD/MM/YYYY username <useremailaddress> > * Changes > > see this . > http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/opendlm/opendlm/ChangeLog?rev=HEAD&content-type=text/plain > > I found this format pretty useful when tracking bugs. I agree with your suggestions. I would also like to insert the following line into the ChangeLog every time I roll a release: DD/MM/YYYY CI x.x.x -- or -- DD/MM/YYYY OpenSSI x.x.x Sound okay? Brian |
From: Aneesh K. K.V <ane...@di...> - 2003-06-03 03:32:27
|
On Tue, 2003-06-03 at 05:52, Brian J. Watson wrote: > Aneesh Kumar K.V wrote: > > I agree with your suggestions. I would also like to insert the following > line into the ChangeLog every time I roll a release: > > DD/MM/YYYY CI x.x.x > -- or -- > DD/MM/YYYY OpenSSI x.x.x > > Sound okay? > Great !!! -aneesh |
From: Brian J. W. <Bri...@hp...> - 2003-06-03 23:55:57
|
Brian J. Watson wrote: > I agree with your suggestions. I would also like to insert the following > line into the ChangeLog every time I roll a release: > > DD/MM/YYYY CI x.x.x > -- or -- > DD/MM/YYYY OpenSSI x.x.x One other suggestion is that we stop including CI changes in the OpenSSI ChangeLog. When I roll OpenSSI releases, I'll just copy in the CI ChangeLog as ChangeLog.ci. We don't need to have the two integrated together. Brian |
From: Brian J. W. <Bri...@hp...> - 2003-06-03 00:33:08
|
Aneesh Kumar K.V wrote: > It would be better if we wait for a day or so and say no more submits > and then reorganise.Then we can get the reorganised RH branch and Trunk > built and then open up the CVS for further submit. To all developers- It's time to implement this reorg. Please don't check anything into the repositories. After I finish updating them, I can help you adapt your sandboxes to the new code organization. Aneesh- Can you help me adapt the configure.ac and Makefile.am files? I'll checkin a first cut at them today, but you're more experienced than I at getting them right. :) -Brian |
From: Aneesh K. K.V <ane...@di...> - 2003-06-03 03:31:03
|
On Tue, 2003-06-03 at 06:00, Brian J. Watson wrote: > Aneesh- > > Can you help me adapt the configure.ac and Makefile.am files? I'll > checkin a first cut at them today, but you're more experienced than I at > getting them right. :) > I will take care of Makefile.am and configure.ac changes. -aneesh |