Re: [Sisuite-devel] SystemImager next generation BIG progress: CentOS-6 and Open SuSE-42.3 support!

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

I forgot to add few points.

If you want to use a main install script (let say for setting IMAGENAME or POST_ACTION, the skeleton can be found in /etc/systemimager/autoinstallscript.template
the most important thing is that it must contain the comment:
# script_version:2
This is to avoid running old script with disk partitionning and other stuffs that couyld break systemimager dracut logic.

A basic script can be:
----------8<----------8<----------8<----------8<----------8<----------8<----------
#!/bin/sh
# script_version:2
# My sample script
#
. /lib/systemimager-lib.sh # load write_variables() function and some log message functions

IMAGENAME=oscar_image
loginfo "Set IMAGENAME=oscar_image"

write_variables # updates /tmp/variables.txt with new IMAGENAME

# end of main script
----------8<----------8<----------8<----------8<----------8<----------8<----------

For the cmdline parameters, here is what I'm using:
si.image-server=10.0.1.1 si.config=testconfig.conf ramdisk_blocksize=1024 ramdisk_size=128000 acpi=no noapic ip=dhcp dns=10.0.1.1 rd.vconsole.keymap=fr vga=791 splash si.debug

And my 10.0.1.1:/var/lib/systemimager/scripts/config/testconfig.conf file:
----------8<----------8<----------8<----------8<----------8<----------8<----------
MAGENAME=testimage
MONITOR_SERVER=10.0.1.1
#MONITOR_PORT=8181
MONITOR_CONSOLE=yes
SKIP_LOCAL_CFG=y
DEBUG=yes
# SCRIPTNAME=testimage.master
# DL_PROTOCOL=rsync
SIS_POST_ACTION=shell
----------8<----------8<----------8<----------8<----------8<----------8<----------

Menu entry in my /var/lib/tftpboot/pxelinux.cfg/default
----------8<----------8<----------8<----------8<----------8<----------8<----------
LABEL systemimager
    MENU LABEL ^S - Install a new Node
    KERNEL sis_kernel
    APPEND initrd=sis_initrd.img root=/dev/ram si.image-server=10.0.1.1 si.config=testconfig.conf ramdisk_blocksize=1024 ramdisk_size=128000 ip=dhcp dns=10.0.1.1 rd.vconsole.keymap=fr vga=791 splash acpi=no noapic
----------8<----------8<----------8<----------8<----------8<----------8<----------

--
   Olivier LAHAYE
   CEA DRT/LIST/DIR
________________________________
De : LAHAYE Olivier
Envoyé : mercredi 21 mars 2018 18:12
À : sis...@li...; sis...@li...
Objet : [PROVENANCE INTERNET] [sisuite-users] SystemImager next generation BIG progress: CentOS-6 and Open SuSE-42.3 support!

Systemimager is still progressing a lot!

New change since last announce:

- Support for CentOS-6 (finaly!)

- Support for Open SuSE-42.3

- Support For CentOS-7

- Support for Fedora-27

- Install script is not mandatory any-more (big change). You can still add one though.
 deployment process is the following:
 pre-install / format disks / mount partitions / run main install script / download/install image / install overrides / create config (mdadm, lvm, fstab, initramfs, ...) / bootloader / post install

- many cmdline parameters to control systemimager imaging process (no more option 140 DHCP parameters to configure (still supported)).
  parameters can be set at multiple places (DHCP, local.cfg, PXE-cmdline, /var/lib/systemimager/scripts/configs/<name>.conf)

- disk layout is set in xml format and stored in /var/lib/systemimager/scripts/disks-layouts/<name>.xml this permits to have a single image deployed to different nodes with different disks. No more one image per disk layout.

- Now protocols are splitted in dracut sub modules (systemimager-xmit)
  * For now, only rsync works. *
  Planned:
  systemimager-xmit-docker: build your image on a docker server (must be complete), install it on deployed hardware!
  systemimager-xmit-flamethrower
  systemimager-xmit-nfs: mount your image tree thru NFS and rsync locally from NFS mount point
  systemimager-xmit-rsync: as usual
  systemimager-xmit-ssh: using scp
  systemimager-xmit-template: a template file for adding more protocols
  systemimager-xmit-torrent: torrent deployment

- SSH tunnel is not considered any-more a protocol for systemimager image deployment.
  SSH tunnel is something running aside like a network layer. It was used for rsync, but IMHO could also be used for other protocols like NFS, docker, ...

- systemimager parameters (set as shell variables in local.cfg or /var/lib/systemimager/scripts/configs/<name>.conf or set as cmdline parameter in PXE) are the following:
  - si.debug / $DEBUG: values: y|n => Enable debug messages
  - si.config / $SIS_CONFIG: name of file containing config (uses less cmdline). File is stored in /var/lib/systemimager/scripts/configs/. Name is given without its PATH
  - si.image-name / $IMAGENAME: name of the image (looks for imagename|imagename.master)
  - si.script-name / $SCRIPTNAME: name of main script (looks for scriptname|scriptname.sh|scriptname.master)
  - si.disk-layout / $DISKS_LAYOUT: name of file containing disks layout in xml format (see man autoinstallscript.conf for now) (looks for filename|filename.xml)
  - si.dl-protocol / $DL_PROTOCOL: protocolname as seen in dracut module systemimager-xmit-<protocol name>.sh. defaults to rsync except if other parameter let think otherwise.
  - si.monitor-server / $MONITOR_SERVER: name or ip for monitor server
  - si.monitor-port / $MONITOR_PORT: monitor server port (defaults to 8181. min:100, max:32000)
  - si.monitor-console / $MONITOR_CONSOLE: y|n|0|1|yes|no (enable/disable log reported to monitor console)
  - si.skip-local-cfg / SKIP_LOCAL_CFG: y|n|0|1|yes|no (default:n) if yes, try to read local.cfg from old disks or floppy (USB planed of course but not yet implemented)
  - si.image-server / $IMAGESERVER: name or ip for the image server
  - si.log-server-port / $LOG_SERVER_PORT: the log server port (default is 514, min:100, max:32000)
  - si.ssh-client / $SSH: y|n|0|1|yes|no (dfault "n").
  - si.ssh-download-url / $SSH_DOWNLOAD_URL: ssh download URL => sets DL_PROTOCOL to "ssh"
  - si.ssh-server / $SSHD: y|n|0|1|yes|no (enable / disable sshd access while imaging) (default "n")
  - si.flamethrower-directory-portbase: $FLAMETHROWER_DIRECTORY_PORTBASE: path
  - si.tmpfs-staging / $TMPFS_STAGING: path to staging dir.
  - si.term / $TERM: terminal type to use. defaults to "linux"
  - si.selinux-relabel / $SEL_RELABEL: y-n|0|1|yes|no: Request a SELinux files relabel after imaging disks
  - si.post-action / $SIS_POST_ACTION: what to do after imaging. defaults to reboot. Possible values: kexec, shell, reboot, shutdown, poweroff (and soon continue)
    continue will boot the imaged system without reboot. This is possible as initramfs is simply mounting OS filesystems in /sysroot and calls swaproot. We are in the same situatiuon after imaging. We only need to check that imaging kernel (uname -r) has modules available in /sysroot/lib/modules/$(uname -r)/, then we exit the dracut hook initqueue and dracut finishes the boot for us.

- Parameters in DHCP are still possible and supported even in wickedd-dhcp4 dhcp client in SuSE distros.
  - option-140: IMAGESERVER
  - option-141: LOG_SERVER_PORT
  - option-142: SSH_DOWNLOAD_URL
  - option-143: FLAMETHROWER_DIRECTORY_PORTBASE
  - option-144: TMPFS_STAGING

IMPORTANT:
- si_mkautoinstallscript is DEPRECATED and shouldn't be used (useless though)
- disk layout has changed. main scripts are now stored in /var/lib/systemimager/scripts/main-install (not /var/lib/systemimager/scripts)
This looks like:
/var/lib/systemimager/scripts/cluster.txt # created by si_clusterconfig from /etc/systemimager/cluster.xml
/var/lib/systemimager/scripts/configs/*.conf
/var/lib/systemimager/scripts/disks-layouts/*.xml
/var/lib/systemimager/scripts/hosts
/var/lib/systemimager/scripts/main-install/*.{,sh,master}
/var/lib/systemimager/scripts/post-install/*.sh
/var/lib/systemimager/scripts/pre-install/*.sh

Packages should be released soon for testing for the 4 distros annouced.

Next:
- support for continueboot after imaging
- *.deb packaging (hopefully if I don't hit dracut conflict with initramfs, seems fixed by now)
  (need a lot of work (major package rewrite and need to compute deps for all perl (automatic for rpm))

All stuffs in initrd-from-imageserver-and-dont-package-initrd form systemimager git

how to build yourself:
git clone -b initrd-from-imageserver-and-dont-package-initrd https://github.com/finley/SystemImager.git
cd systemImager
make source_tarball
rpmbuild -tb tmp/systemimager-4.5.0.tar.gz
yum|dnf|zypper install <list of built package except flamethrower and bittorrent one>

Happy testing.

PS: Please report any issues.
--
   Olivier LAHAYE