crucible-devel — Development discussion

Re: [Crucible-devel] [Autotest] Re: Attempt to add boot-once support for GRUB in boottool

[Randy D. <rd...@xe...>]

On Mon, 25 Jun 2007 16:23:00 -0700 Ryan Stutsman wrote:

> Bryce,
> 
> I think there was some hesitance to use the modularized version of
> boottool for the sake of simplicity, but after looking at the INSTALL
> file method for generating a monolithic script we want to have
> autotest use the updated version.
> 
> I noticed one issue so far using the new script.  For the 'default'
> command in menu.lst when it is set to an integer and not 'saved'
> running set-default gobbles the newline and the resulting menu.lst
> contains the correct default entry but with the following line pulled
> up into it.
> 
> Adding a \n to the regexp replace in the Grub.pm fixed it for me.  Any
> idea if this is a bug or am I simply doing something wrong?
> 
> Here is the diff to clarify what I changed:
> 
> Index: lib/Linux/Bootloader/Grub.pm
> ===================================================================
> --- lib/Linux/Bootloader/Grub.pm        (revision 735)
> +++ lib/Linux/Bootloader/Grub.pm        (working copy)
> @@ -227,7 +228,7 @@
>    foreach my $index (0..$#config) {
> 
>      if ($config[$index] =~ /(^\s*default\s*\=*\s*)\d+/i) {
> -      $config[$index] = "$1$newdefault";
> +      $config[$index] = "$1$newdefault\n";
>        last;
>      } elsif ($config[$index] =~ /^\s*default\s*\=*\s*saved/i) {
>        my @default_config;
> 
> 
> -Ryan

Applied to Linux-Bootloader SVN also.  Thanks.

---
~Randy
Features and documentation: http://lwn.net/Articles/260136/

Re: [Crucible-devel] [patch] do kernel builds using $BUILDARCH

[Bryce H. <br...@br...>]

Sounds great.

On Tue, Dec 04, 2007 at 03:34:21PM -0800, Randy Dunlap wrote:
> I'm committing a patchset with this description:
> 
> 
> Build using BUILDARCH if it is set (e.g., via export in a plan file).
> This allows for both i386/x86_64 builds.
> Does not support full cross-compiles nor UML.
> 
> For allno/yes/modconfig builds, build output is produced in numbered
> files, e.g.,
>   runs/2211/logs/caor-control/kernel.make.log.i386.~[123]
> with ~1 for allnoconfig, ~2 for allyesconfig, and ~3 for allmodconfig.
> 
> 
> and diffstat:
>  testing/plans/commands/build_only_kernel          |    5 +-
>  testing/plans/commands/create_kernel_allmodconfig |   12 ++++-
>  testing/plans/commands/create_kernel_allnoconfig  |   12 ++++-
>  testing/plans/commands/create_kernel_allyesconfig |   12 ++++-
>  testing/plans/commands/create_kernel_defconfig    |   12 ++++-
>  scripts/build_kernel                      |   48 ++++++++++++--------
>  6 files changed, 75 insertions(+), 26 deletions(-)
> 
> ---
> ~Randy
> Features and documentation: http://lwn.net/Articles/260136/

[Crucible-devel] [patch] do kernel builds using $BUILDARCH

[Randy D. <rd...@xe...>]

I'm committing a patchset with this description:


Build using BUILDARCH if it is set (e.g., via export in a plan file).
This allows for both i386/x86_64 builds.
Does not support full cross-compiles nor UML.

For allno/yes/modconfig builds, build output is produced in numbered
files, e.g.,
  runs/2211/logs/caor-control/kernel.make.log.i386.~[123]
with ~1 for allnoconfig, ~2 for allyesconfig, and ~3 for allmodconfig.


and diffstat:
 testing/plans/commands/build_only_kernel          |    5 +-
 testing/plans/commands/create_kernel_allmodconfig |   12 ++++-
 testing/plans/commands/create_kernel_allnoconfig  |   12 ++++-
 testing/plans/commands/create_kernel_allyesconfig |   12 ++++-
 testing/plans/commands/create_kernel_defconfig    |   12 ++++-
 scripts/build_kernel                      |   48 ++++++++++++--------
 6 files changed, 75 insertions(+), 26 deletions(-)

---
~Randy
Features and documentation: http://lwn.net/Articles/260136/

[Crucible-devel] [patch] Linux::Bootloader updates

[Randy D. <rd...@xe...>]

Hi,

I've just checked in the attached 2 patches to Linux::Bootloader.

- linux_install: support for arch/x86/;
- linux_mkinitrd: use 'distro_id' script to know how
  to pass params to mkinitrd;

---
~Randy

[Crucible-devel] [patch] allow using kexec to "reboot" into a test kernel

[Randy D. <rd...@xe...>]

Add support for using kexec to boot into a new test kernel.
It's quick and eliminates need for using grub/lilo etc.

Fix use of $kernel_label in boot_kernel initrd filename; it should be
$kernel_id instead.
Previous patch forgot to pass $initrd to boottool, so do that now.
Use corrected initrd filename.

Minor:
Add help/usage text in usr/bin/build_kernel.
Change any if [ ... == ...] to use only one = sign (in these 3 scripts).

---
 plans/commands/boot_kernel             |   41 ++++++++++++++++++----
 plans/commands/build_miniconfig_kernel |    6 ++-
 usr/bin/build_kernel                   |   60 ++++++++++++++++++++++-----------
 3 files changed, 77 insertions(+), 30 deletions(-)

--- cruc.orig/plans/commands/boot_kernel
+++ cruc/plans/commands/boot_kernel
@@ -10,20 +10,22 @@ SUT=`hostname -s`
 initrd=""
 if [ "$kconfig" != "" ]; then
 	if grep "CONFIG_BLK_DEV_INITRD=y" $kconfig >/dev/null 2>&1; then
-		filename="initrd-$kernel_label.img"
+		filename="/boot/initrd-$kernel_id.img"
 		initrd="--initrd=$filename"
 		echo "using initrd, filename:$filename"
 	fi
 fi
 
-echo boottool --boot-once --title "$kernel_label" "$initrd"
-boottool --boot-once --title "$kernel_label"
-err=$?
+if [ "$kexec_reboot" != "1" ]; then
+    echo boottool --boot-once --title "$kernel_label" "$initrd"
+    boottool --boot-once --title "$kernel_label" "$initrd"
+    err=$?
 
-if [ $err -ne 0 ]; then
-    echo "Error:  Lilo failed to set next kernel."
-    echo "EXIT STATUS(lilo):  $err"
-    return $err
+    if [ $err -ne 0 ]; then
+	echo "Error:  Lilo failed to set next kernel."
+	echo "EXIT STATUS(lilo):  $err"
+	return $err
+    fi
 fi
 
 # set autoreboot on kernel panic
@@ -36,6 +38,29 @@ echo Powercycling machine
 umount -la -t nfs
 umount -la -t nfs4
 
+if [ "$kexec_reboot" = "1" ]; then
+# all kexec loading of the new/test kernel & passing control to it are handled here
+
+	# can add to cmdline as needed:
+	cmdline="$kernel_args"
+	echo "kexec reboot wants to use: kernel=/boot/kernel-$kernel_id + initrd=$initrd + cmdline=$cmdline"
+	kexec --load /boot/kernel-$kernel_id $initrd --append="$cmdline"
+	kex=$?
+	if [ $kex -ne 0 ]; then
+		echo "kexec load failed, error = $kex"
+		exit -10
+	fi
+	echo "kexec loaded new kernel image, now exec it:"
+	sleep 1
+	kexec --exec
+	kex=$?
+	if [ $kex -ne 0 ]; then
+		echo "kexec exec new kernel failed, error =  $kex"
+		exit -11
+	fi
+	exit 0
+fi
+
 echo /sbin/shutdown now -r
 /sbin/shutdown now -r
 err=$?
--- cruc.orig/usr/bin/build_kernel
+++ cruc/usr/bin/build_kernel
@@ -15,7 +15,10 @@ source $BASE_DIR/plans/commands/common.f
 # Variables
 PATH=$BASE_DIR/usr/bin:/sbin:/usr/sbin:$PATH
 
+echo "start build_kernel: with args=$@"
+
 build_only=0
+kexec_reboot=0
 OPT_K=''
 src_dir=${SRC_DIR:-/usr/src}
 default_config=${DEFAULT_CONFIG:-$PKGS_DIR/linux/config.default}
@@ -25,11 +28,11 @@ karch=`echo $arch | sed -e s/i.86/i386/ 
      -e s/s390x/s390/ -e s/sa110/arm/ -e s/ppc64/powerpc/ -e s/parisc64/parisc/`
 
 usage() {
-    echo "USAGE:  $0 -i <kernel-id> [-k kernel-label] [-c config-default] [-a kernel-args] [-l log-dir]"
+    echo "USAGE:  $0 -i <kernel-id> [-k kernel-label] [-c config-default] [-a kernel-args] [-l log-dir] [-b] [-x]"
     exit -1
 }
 
-while getopts ":i:k:c:a:l:b" opt; do
+while getopts ":i:k:c:a:l:bx" opt; do
     case $opt in
     i ) id=$OPTARG ;;
     k ) label=$OPTARG ;;
@@ -37,6 +40,7 @@ while getopts ":i:k:c:a:l:b" opt; do
     a ) kernel_args=$OPTARG ;;
     l ) log_dir=$OPTARG ;;
     b ) build_only=1 ; OPT_K="-k" ;;
+    x ) kexec_reboot=1 ;;
     \? ) usage ;;
     esac
 done
@@ -61,7 +65,7 @@ fi
 if [ "$kernel_args" ]; then 
   kernel_args="--args=$kernel_args"
 fi
-echo "Using args: id: $id kernel: $kernel label: $label kargs: $kernel_args"
+echo "Using args: id: $id; kernel: $kernel; label: $label; kargs: $kernel_args"
 
 # Strip off any leading "linux-"
 id=${id#linux-}
@@ -122,6 +126,15 @@ kconfig_number=`get_next_file_number $lo
 cp -a .config $log_dir/kconfig.~$kconfig_number
 echo "Saved .config as $log_dir/kconfig.~$kconfig_number"
 
+# remember whether this .config uses an initrd:
+if grep "CONFIG_BLK_DEV_INITRD=y" .config >/dev/null 2>&1; then
+	use_initrd=1
+	echo "kconfig uses initrd"
+else
+	use_initrd=0
+	echo "kconfig: no initrd"
+fi
+
 ##export this config file name
 # add "kconfig=filename" to run_profile file for next step(s)
 rundir=$log_dir
@@ -138,27 +151,32 @@ echo "make $OPTS $OPT_K all"
 make $OPTS $OPT_K all >> $log_dir/kernel.make.log 2>&1
 sts=$?
 [ $sts -ne 0 ] && echo "ERROR: command failed"
-[ $sts -ne 0 ] && [ "$build_only" == "0" ] && exit -12
+[ $sts -ne 0 ] && [ "$build_only" = "0" ] && exit -12
 
-if [ "$build_only" == "0" ]; then
+if [ "$build_only" = "0" ]; then
     if grep "CONFIG_MODULES=y" .config >/dev/null 2>&1; then
         echo "make modules modules_install"
         make modules modules_install >> $log_dir/kernel.modules.log 2>&1
 	sts=$?
         [ $sts -ne 0 ] && echo "ERROR: command failed"
-        [ $sts -ne 0 ] && [ "$build_only" == "0" ] && exit -14
+        [ $sts -ne 0 ] && [ "$build_only" = "0" ] && exit -14
     else
         echo "skipping modules_install: not enabled"
     fi
 fi
 
-if [ "$build_only" == "0" ]; then
+# copy kernel to /boot even for kexec_reboot since the build dir may be wiped out,
+# but don't add it to the boot config file
+if [ "$build_only" = "0" ]; then
     echo "Copying kernel to /boot"
+    echo "names: id=$id, src_dir=$src_dir, karch=$karch"
     linux_install $id $src_dir/linux-$id /boot $karch >> $log_dir/kernel.install.log 2>&1
     [ $? -ne 0 ] && echo "ERROR: command failed" && exit -16
 
-    # Create a new initrd
-    #linux_mkinitrd $id /boot >> $log_dir/kernel.initrd.log 2>&1
+    if [ $use_initrd -eq 1 ]; then
+	# Create a new initrd
+	linux_mkinitrd $id /boot >> $log_dir/kernel.initrd.log 2>&1
+    fi
 fi
 
 # Cleanup the kernel source tree to save disk space
@@ -167,17 +185,19 @@ make clean >> $log_dir/kernel.clean.log 
 [ $? -ne 0 ] && echo "WARNING: command failed, but continuing"
 
 # finished for build_only case:
-[ "$build_only" == "1" ] && exit 0
+[ "$build_only" = "1" ] && exit 0
+
+# finished for kexec_reboot case:
+[ "$kexec_reboot" = "1" ] && exit 0
 
-# Remove a kernel if more than 10 already
+# Remove a kernel if more than 10 already (not used for kexec_reboot)
 kcount=`boottool --info=all | grep -c index`;
 if [ $kcount -gt 10 ]; then
-  echo "More than 10 kernels.  Removing one from middle..."
-  boottool --remove-kernel 5
+    echo "More than 10 kernels.  Removing one from middle..."
+    boottool --remove-kernel 5
 fi
 
-# Add a section to bootloader for new kernel
-
+# Add a section to bootloader for new kernel (not used for kexec_reboot)
 for name in kernel vmlinuz vmlinux; do
     if [ -e /boot/$name-$id ]; then
         kernel="/boot/$name-$id"
@@ -186,12 +206,12 @@ for name in kernel vmlinuz vmlinux; do
 done
 
 echo "Updating bootloader config"
-if [ -f /boot/initrd.img-$id ]; then
-  boottool --add-kernel=$kernel --title="$label" --savedefault 0 \
-    --initrd=/boot/initrd.img-$id --position=end "$kernel_args" --force
+if [ $use_initrd -eq 1 ]; then
+    boottool --add-kernel=$kernel --title="$label" --savedefault 0 \
+	--initrd=/boot/initrd-$id.img --position=end "$kernel_args" --force
 else
-  boottool --add-kernel=$kernel --title="$label" "$kernel_args" --savedefault 0 \
-    --position=end --force
+    boottool --add-kernel=$kernel --title="$label" "$kernel_args" --savedefault 0 \
+	--position=end --force
 fi
 [ $? -ne 0 ] && echo "ERROR: command failed" && exit -20
 
--- cruc.orig/plans/commands/build_miniconfig_kernel
+++ cruc/plans/commands/build_miniconfig_kernel
@@ -5,7 +5,7 @@
 . $base_dir/plans/commands/common.fns
 
 [ -f $base_dir/etc/global.cfg ] && source $base_dir/etc/global.cfg
-if [ "$SRC_DIR" == "" ]; then
+if [ "$SRC_DIR" = "" ]; then
 	SRC_DIR="/usr/local/src"
 fi
 
@@ -59,10 +59,12 @@ cd $src_dir
 KCONFIG_ALLCONFIG=$default_config make allnoconfig >$log_dir/kernel.miniconfig.log 2>&1
 [ ! -e $src_dir/.config ] && echo "ERROR: no .config created" && exit 1
 
+kexec_arg=""
+[ $kexec_reboot = "1" ] && kexec_arg="-x"
 [ $kernel_id ] && kernel_id="-i $kernel_id"
 [ $kernel_label ] && kernel_label="-k $kernel_label"
 [ $default_config ] && default_config="-c $default_config"
 [ "$kernel_args" ] && kernel_args="-a $kernel_args"
 [ $log_dir ] && log_dir="-l $log_dir"
 
-build_kernel $kernel_id $kernel_label $default_config $log_dir "$kernel_args"
+build_kernel $kernel_id $kernel_label $default_config $log_dir $kexec_arg "$kernel_args"

[Crucible-devel] [patch] handle arch/x86/ merger

[Randy D. <rd...@xe...>]

Both Linux-Bootloader::linux_install and Crucible::build_kernel
need to know about, allow, and support the merger of arch/i386/
and arch/x86_64/ into arch/x86/.  Here are the patches that I am
currently using for this.  After a little more testing, I'll
check these (or fixes to them) into the sf.net svn repo unless
I hear comments about them...

---
~Randy

Re: [Crucible-devel] [patch 1/2] build_kernel: publish kernel config filename for later

[Bryce H. <br...@br...>]

No comments, they look fine!

On Fri, Oct 12, 2007 at 02:25:11PM -0700, Randy Dunlap wrote:
> Any comments before I merge these 2 patches?
> 
> 
> ---
> 
> 
> Put the kernel config filename into run_profile.txt so that later scripts
> (e.g., boot_kernel) can see/use the config file.
> 
> Uses 'basedrop' script, attached, which will be merged into usr/bin
> along with some other utility scripts.
> 
> 
> ---
>  scripts/build_kernel |   11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> --- cruc.orig/scripts/build_kernel
> +++ cruc/scripts/build_kernel
> @@ -112,6 +112,17 @@ kconfig_number=`get_next_file_number $lo
>  cp -a .config $log_dir/kconfig.~$kconfig_number
>  echo "Saved .config as $log_dir/kconfig.~$kconfig_number"
>  
> +##export this config file name
> +# add "kconfig=filename" to run_profile file for next step(s)
> +rundir=$log_dir
> +# now drop 2 trailing levels of rundir to find run_profile.txt
> +rundir=`basedrop $rundir`
> +rundir=`basedrop $rundir`
> +profile="$rundir/run_profile.txt"
> +echo "profile filename = $profile"
> +echo "publish kconfig=$log_dir/kconfig.~$kconfig_number"
> +echo "kconfig=\"$log_dir/kconfig.~$kconfig_number\"" >>$profile
> +
>  # "all" builds kernel + modules
>  echo "make $OPTS $OPT_K all"
>  make $OPTS $OPT_K all >> $log_dir/kernel.make.log 2>&1

[Crucible-devel] [patch 2/2] boot_kernel: pass along "--initrd"

[Randy D. <rd...@xe...>]

Pass "--initrd <initrd.img filename>" parameter to boot_kernel
if CONFIG_BLK_DEV_INITRD=y in the build config file.

---
 testing/plans/commands/boot_kernel |   11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

--- cruc.orig/testing/plans/commands/boot_kernel
+++ cruc/testing/plans/commands/boot_kernel
@@ -7,7 +7,16 @@ kernel_info
 
 SUT=`hostname -s`
 
-echo boottool --boot-once --title "$kernel_label"
+initrd=""
+if [ "$kconfig" != "" ]; then
+	if grep "CONFIG_BLK_DEV_INITRD=y" $kconfig >/dev/null 2>&1; then
+		filename="initrd-$kernel_label.img"
+		initrd="--initrd $filename"
+		echo "using initrd, filename:$filename"
+	fi
+fi
+
+echo boottool --boot-once --title "$kernel_label" "$initrd"
 boottool --boot-once --title "$kernel_label"
 err=$?
 

[Crucible-devel] [patch 1/2] build_kernel: publish kernel config filename for later

[Randy D. <rd...@xe...>]

Any comments before I merge these 2 patches?


---


Put the kernel config filename into run_profile.txt so that later scripts
(e.g., boot_kernel) can see/use the config file.

Uses 'basedrop' script, attached, which will be merged into usr/bin
along with some other utility scripts.


---
 scripts/build_kernel |   11 +++++++++++
 1 file changed, 11 insertions(+)

--- cruc.orig/scripts/build_kernel
+++ cruc/scripts/build_kernel
@@ -112,6 +112,17 @@ kconfig_number=`get_next_file_number $lo
 cp -a .config $log_dir/kconfig.~$kconfig_number
 echo "Saved .config as $log_dir/kconfig.~$kconfig_number"
 
+##export this config file name
+# add "kconfig=filename" to run_profile file for next step(s)
+rundir=$log_dir
+# now drop 2 trailing levels of rundir to find run_profile.txt
+rundir=`basedrop $rundir`
+rundir=`basedrop $rundir`
+profile="$rundir/run_profile.txt"
+echo "profile filename = $profile"
+echo "publish kconfig=$log_dir/kconfig.~$kconfig_number"
+echo "kconfig=\"$log_dir/kconfig.~$kconfig_number\"" >>$profile
+
 # "all" builds kernel + modules
 echo "make $OPTS $OPT_K all"
 make $OPTS $OPT_K all >> $log_dir/kernel.make.log 2>&1

Re: [Crucible-devel] [osdldbt-general] Test::Parser getting errors on install

[Bryce H. <br...@br...>]

On Thu, Aug 16, 2007 at 08:47:32AM -0700, Mark Wong wrote:
> Hi Kate, Bryce,
> 
> On 8/16/07, Coyne, Kate <Kat...@st...> wrote:
> > Hello,
> >  I've attempted to install cpan -i Test::Parser manually and auto but
> > get the same results.  It doesn't install due to too many errors.  I
> > went to the site and installed all the prerequisites in the Makefile.PL
> > myself(just to be sure) but still got the same results: See below. Am I
> > missing something on the install?
> > It doesn't automatically install due to the errors though one could
> > force it too..but I'm not going to do that without looking into it
> > beforehand. Also if I run a workload I get errors in the processing
> > results area!(which lead me here again).
> 
> Bryce, any ideas?  All the tests that failed aren't pieces I
> contributed so I'm at a loss...

Don't know, maybe try running them individually and seeing what the
actual error messages are?  E.g.,

  perl t/01_script_compile

It's odd that the error it's giving is 'dubious'.  Haven't seen that
before.

Bryce
 
> > Here's the output:
> [snip]
> > Running make test
> > PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e"
> > "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
> > t/00_initialize........ok
> > t/01_script_compile....dubious
> >         Test returned status 3 (wstat 768, 0x300)
> > DIED. FAILED tests 2-3, 6
> >         Failed 3/10 tests, 70.00% okay
> > t/02_kernel_build......ok
> > t/03_ltp...............dubious
> >         Test returned status 2 (wstat 512, 0x200)
> > DIED. FAILED tests 1-5
> >         Failed 5/5 tests, 0.00% okay
> > t/04_lhcs..............dubious
> >         Test returned status 2 (wstat 512, 0x200)
> > DIED. FAILED tests 1-5
> >         Failed 5/5 tests, 0.00% okay
> > t/05_newpynfs..........dubious
> >         Test returned status 2 (wstat 512, 0x200)
> > DIED. FAILED tests 1-5
> >         Failed 5/5 tests, 0.00% okay
> > t/06_iostat............ok
> > t/07_oprofile..........ok
> > t/08_pgoptions.........ok
> > t/09_readprofile.......ok
> > t/10_sar...............ok
> > t/11_sysctl............ok
> > t/12_vmstat............ok
> > t/13_iozone............dubious
> >         Test returned status 2 (wstat 512, 0x200)
> > DIED. FAILED tests 1-2
> >         Failed 2/2 tests, 0.00% okay
> > t/zz_dump_config.......ok
> > Failed Test           Stat Wstat Total Fail  Failed  List of Failed
> > ------------------------------------------------------------------------
> > -------
> > t/01_script_compile.t    3   768    10    3  30.00%  2-3 6
> > t/03_ltp.t               2   512     5   10 200.00%  1-5
> > t/04_lhcs.t              2   512     5   10 200.00%  1-5
> > t/05_newpynfs.t          2   512     5   10 200.00%  1-5
> > t/13_iozone.t            2   512     2    4 200.00%  1-2
> >   /usr/bin/make test -- NOT OK
> > Running make install
> >   make test had returned bad status, won't install without force
> > Thanks again!
> > Kate
> >
> >
> >
> > -------------------------------------------------------------------------
> > This SF.net email is sponsored by: Splunk Inc.
> > Still grepping through log files to find problems?  Stop.
> > Now Search log events and configuration files using AJAX and a browser.
> > Download your FREE copy of Splunk now >>  http://get.splunk.com/
> > _______________________________________________
> > osdldbt-general mailing list
> > osd...@li...
> > https://lists.sourceforge.net/lists/listinfo/osdldbt-general
> >

Re: [Crucible-devel] [osdldbt-general] Test::Parser getting errors on install

[Mark W. <ma...@gm...>]

Hi Kate, Bryce,

On 8/16/07, Coyne, Kate <Kat...@st...> wrote:
> Hello,
>  I've attempted to install cpan -i Test::Parser manually and auto but
> get the same results.  It doesn't install due to too many errors.  I
> went to the site and installed all the prerequisites in the Makefile.PL
> myself(just to be sure) but still got the same results: See below. Am I
> missing something on the install?
> It doesn't automatically install due to the errors though one could
> force it too..but I'm not going to do that without looking into it
> beforehand. Also if I run a workload I get errors in the processing
> results area!(which lead me here again).

Bryce, any ideas?  All the tests that failed aren't pieces I
contributed so I'm at a loss...

> Here's the output:
[snip]
> Running make test
> PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e"
> "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
> t/00_initialize........ok
> t/01_script_compile....dubious
>         Test returned status 3 (wstat 768, 0x300)
> DIED. FAILED tests 2-3, 6
>         Failed 3/10 tests, 70.00% okay
> t/02_kernel_build......ok
> t/03_ltp...............dubious
>         Test returned status 2 (wstat 512, 0x200)
> DIED. FAILED tests 1-5
>         Failed 5/5 tests, 0.00% okay
> t/04_lhcs..............dubious
>         Test returned status 2 (wstat 512, 0x200)
> DIED. FAILED tests 1-5
>         Failed 5/5 tests, 0.00% okay
> t/05_newpynfs..........dubious
>         Test returned status 2 (wstat 512, 0x200)
> DIED. FAILED tests 1-5
>         Failed 5/5 tests, 0.00% okay
> t/06_iostat............ok
> t/07_oprofile..........ok
> t/08_pgoptions.........ok
> t/09_readprofile.......ok
> t/10_sar...............ok
> t/11_sysctl............ok
> t/12_vmstat............ok
> t/13_iozone............dubious
>         Test returned status 2 (wstat 512, 0x200)
> DIED. FAILED tests 1-2
>         Failed 2/2 tests, 0.00% okay
> t/zz_dump_config.......ok
> Failed Test           Stat Wstat Total Fail  Failed  List of Failed
> ------------------------------------------------------------------------
> -------
> t/01_script_compile.t    3   768    10    3  30.00%  2-3 6
> t/03_ltp.t               2   512     5   10 200.00%  1-5
> t/04_lhcs.t              2   512     5   10 200.00%  1-5
> t/05_newpynfs.t          2   512     5   10 200.00%  1-5
> t/13_iozone.t            2   512     2    4 200.00%  1-2
>   /usr/bin/make test -- NOT OK
> Running make install
>   make test had returned bad status, won't install without force
> Thanks again!
> Kate
>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> _______________________________________________
> osdldbt-general mailing list
> osd...@li...
> https://lists.sourceforge.net/lists/listinfo/osdldbt-general
>

[Crucible-devel] new crucible checkins

[Randy D. <rd...@xe...>]

Hi all :)

I've just checked in several Crucible scripts that I use on a
regular basis.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
09-netconsole:

Add a report script to kill netconsole loggers (netcat) at the end
of a test run.  Also checks for Oops and panics and prints a message
about that if found.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
posixtest-summary.sh:

Scan an Open Posix Test suite output log looking for PASS, FAILED,
UNTESTED, UNRESOLVED, UNSUPPORTED, INTERRUPTED, and build errors,
then compute a SCORE value based on those numbers.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
save_sysinfo:

dumps SUT info and many /proc files to a sequentially-numbered
log directory (since this is meant to be used many times per
test run).
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
grab_patch_kernel:

Downloads a base kernel (like linux-2.6.20.tar.bz2) plus patches
(including -rcN, -gitM, -mmX) and applies all rc/git/mm patches.
Also applies patches in the environment variable "patchdir" if
that variable is non-null.  All patches in "patchdir" are applied
in sorted order (number/alpha order).  It is up to the patchdir
creator to make sure that the files are named so that they will
apply in the required order.  One easy way to do this is to name
them like:  001-kernel-sys.patch, 002-dcache-panic.patch,
003-usb-storage.patch.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rm_kernel_tree:

Can be used in plan 'commands' to remove a kernel tree after the
kernel has been built and installed.  Useful for keeping disk space
usage down when space is critical.  Automates this housekeeping
duty so that it does not have to be done manually on a regular basis.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
clean_out_boot.sh:

For lilo users, this script can be used to clean /boot of kernels
that have been used for testing but don't need to stay in /boot
for a long time.  Depending on the kernel that was booted,
clean_out_boot.sh removes other kernels (-git kernels removes
-mm kernels, -mm kernels removes -git kernels, -rc kernels
remove both -git and -mm kernels, and non-suffix kernels
remove -mm, -git, and -rc kernels).  All kernels that are removed
also have their associated System.map and config files removed.

This script should be called from an init script, such as
/etc/init.d/boot.local.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cpus_online.sh:

Returns the number of CPUS online.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
grab_kernel:

Downloads a base kernel and patches (either -rc, -mm, or -git)
and applies the patches to the base kernel.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
memsize.sh:

Returns the system memory size in MB.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
netcon_kill_log:

Kills 'netcat' processes that are running.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
netcon_start_log:

Starts a netcat process for a specific SUT and captures the log
data to $logdir/$logfile.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
procfsdump:

Dumps many /proc files to stdout.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
prtag2tag.pl:

Reads stdin and prints all lines that are between (and including)
begin_tag and end_tag.  The begin & end tags can occur anywhere
in a line.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rebuild_lilo_conf:

Meant to be run following clean_out_boot.sh, it rewrites the
/etc/lilo.conf file based on kernel images that are remaining and
it omits kernel images that have been removed.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
timelmt:

Limits the amount of time that a named program can run.
Time is entered in minutes only.
First start the program, e.g., fsx:  fsx &
Then run:  timelmt 30 fsx
If the program is still running after T minutes, it will be killed.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

---
~Randy

Re: [Crucible-devel] Attempt to add boot-once support for GRUB in boottool

[Bryce H. <br...@br...>]

Hi Ryan,

I wonder if the boottool script you're using is out of date, as there is
already grub boot-once support implemented.  Could you check out the
original source for this in subversion, and produce your patch against
it (if still needed)?

 svn co https://crucible.svn.sourceforge.net/svnroot/crucible/Linux-Bootloader Linux-Bootloader

Bryce

On Mon, Jun 25, 2007 at 12:29:14PM -0700, Ryan Stutsman wrote:
> Hello,
> 
> I've written an attempt at boot-once support for GRUB for boottool and
> I'd be hesitant to use it without some review (particularly given my
> Perl skills).
> 
> I've done some testing with it on a couple of machines.
> 
> There is one awkward constraint that is difficult to get around.  The
> kernel being specified to boot once must have a 'savedefault' command
> entry (it doesn't matter what argument it has).
> 
> Basically, this code will adjust the savedefault entry for the
> specified kernel to be set to the index of the kernel that was
> previously the default.  It then makes the specified kernel the
> default.
> 
> Please let me know if there are any problems.
> 
> After boottool has been run to add a kernel boot once is like this:
> 
> boottool --title=autotest --boot-once
> 
> -Ryan

> Index: client/tools/boottool
> ===================================================================
> --- client/tools/boottool	(revision 540)
> +++ client/tools/boottool	(working copy)
> @@ -468,6 +468,64 @@
>      #return 1;
>  }
>  
> +# Set kernel to be booted once
> +
> +# boot-once for grub requires a bit of setup:
> +#     the kernel that is to be booted once must have a savedefault
> +#     command line even if it has no arguments
> +# This script will set the specified kernel to be default, when
> +# the specified kernel is booted grub will revert the default
> +# to what it was before this script was run (e.g. the specified
> +# kernel entry will have a savedefault entry with the previous default
> +
> +sub boot_once_grub {
> +    my $label = shift;
> +
> +    return undef unless defined $label;
> +
> +    &read( $params{config_file} );
> +    my @sections = &_info();
> +    my $prior_default = &get_default();
> +
> +    my @config = @bootconfig;
> +    my $index = 0;
> +    my $found = 0;
> +    # Skim the configuration file until we find the label of the kernel
> +    # to mark as boot once
> +    foreach (@config) {
> +        # Ensure that default is set to 'saved'
> +        if ( $index == 0 ) {
> +            $_ =~ s/^(\s*)default.*$/$1default saved/i;
> +        }
> +        # Increase index if we are iterating through the next kernel
> +        if ( $_ =~ /^\s*title/i ) {
> +            $index++;
> +        }
> +        # If we find the kernel record which position at which it sits
> +        if ( $_ =~ /^\s*title\s*$label\s/ ) {
> +            $found = $index;
> +        }
> +        # If we have found the kernel try to adjust the saveddefault entry
> +        if ( $found && $_ =~ /^\s*savedefault.*$/) {
> +            # If $found != $index we missed the savedefault entry
> +            ( $found == $index ) ||
> +                die ("No savedefault found in the specified kernel entry.\n");
> +            # Change the line to restore the default when this kern boots
> +            $_ =~ s/^(\s*)savedefault.*$/$1savedefault $prior_default/i;
> +            last;
> +        }
> +    }
> +    $found || die ("No such kernel entry.");
> +    # this needs to be done just before setting default so that
> +    # set_default will handle this by writing to the default file
> +    @bootconfig = @config;
> +    # If everything went well just set the $found - 1 kernel to default
> +    &set_default($found - 1);
> +    &write( $params{config_file} );
> +
> +    return 1;
> +}
> +
>  ### LILO functions ###
>  
>  # Run command to install bootloader
> @@ -1135,6 +1193,8 @@
>          &boot_once_lilo( $params{title} );
>      } elsif ( $detected_bootloader eq 'elilo' ) {
>          &boot_once_elilo( $params{title} );
> +    } elsif ( $detected_bootloader eq 'grub' ) {
> +        &boot_once_grub( $params{title} );
>      } else {
>          print "$detected_bootloader does not have boot-once support.\n";
>          print "Setting as default instead.\n";

Re: [Crucible-devel] Correction to grub default patch

[Bryce H. <br...@br...>]

Thanks, applied.

On Mon, Jun 25, 2007 at 10:32:19AM -0700, Ryan Stutsman wrote:
> Hello,
> 
> I have a small correction to last week's default command comment
> patch.  The original version caused a buildup of comments in certain
> cases in the conf file.
> 
> I've simply removed the comment in this patch.
> 
> -Ryan

> Index: client/tools/boottool
> ===================================================================
> --- client/tools/boottool	(revision 540)
> +++ client/tools/boottool	(working copy)
> @@ -233,7 +233,7 @@
>  
>      foreach my $index ( 0 .. $#config ) {
>          if ( $config[$index] =~ /(^\s*default\s*\=*\s*)\d+/i ) {
> -            $config[$index] = "# set by $0\n$1$newdefault\n";
> +            $config[$index] = "$1$newdefault\n";
>              last;
>          }
>          elsif ( $config[$index] =~ /^\s*default\s*\=*\s*saved/i ) {

[Crucible-devel] Announce Linux::Bootloader 1.3

[Bryce H. <br...@br...>]

Thanks Christian,

I've added both these patches and cut a new release 1.3.  It should be
available on CPAN within the next few hours.

Bryce

On Tue, May 08, 2007 at 10:09:07AM +0200, Keck, Christian (ext) wrote:
> Hello Bryce Harrington,
>  
> I just modified your Linux::Bootloader perl module, because there was no
> grub-parameter for "module" in your Grub.pm. 
>  
> Example (menu.lst): 
>     title Xen (2.6.16.33-xen_3.0.4.1)
>             root (hd0,0)
>             kernel /xen-3.0.4.1.gz
>             module /vmlinuz-2.6.16.33-xen_3.0.4.1 ro root=LABEL=/1
>             module /initrd-2.6.16.33-xen_3.0.4.1.img
>  
> Attached you'll find patches for Grub.pm and boottool (which i had to
> adjust to the new parameter). 
> If you agree with it, please add it to cpan repository.
>  
> Best regards,
> Christian Keck
> 
>  

Re: [Crucible-devel] state of crucible

[Bryce H. <br...@br...>]

On Mon, Apr 23, 2007 at 06:30:27PM -0700, Jon Phillips wrote:
> What is the state of crucible Bryce? And, also, the state post
> OSDL/linux foundation, etc? I'm curious what bearing that has on this
> project and the usage by others :) Also, how can we all help promote
> this project, etc? Will it get used in any new gigs ;)

Randy has been using it at Oracle.  Leann sent a couple patches about a
month ago so I'm guessing she at least looked at using it at Lightfleet.

For Xorg testing, my long term plan is to assemble a variety of hardware
and use Crucible to do automated Xorg testing.  However, near term most
of my projects will just need eyeball testing, and won't benefit from
automation.  I don't have a strong idea of a timeframe for this yet.
Meanwhile, I'm happy to continue reviewing/integrating patches from
other folks using it.

The best way to help right now is to go through an installation of it,
set it up to run some test you care about, and send in patches to fix
any breakages you find.  A good PR step would be to then write up an
article describing what you did and how it proved helpful, and get it
published on some Linux zine site somewhere.  :-)

Bryce

[Crucible-devel] state of crucible

[Jon P. <jo...@re...>]

What is the state of crucible Bryce? And, also, the state post
OSDL/linux foundation, etc? I'm curious what bearing that has on this
project and the usage by others :) Also, how can we all help promote
this project, etc? Will it get used in any new gigs ;)

Jon

-- 
Jon Phillips

San Francisco, CA
USA PH 510.499.0894
jo...@re...
http://www.rejon.org

MSN, AIM, Yahoo Chat: kidproto
Jabber Chat: re...@gr...
IRC: re...@ir...

[Crucible-devel] New releases for fixes to DBT-5 modules.

[Mark W. <ma...@gm...>]

Hi Bryce,

Can you make another release of Test-Parser and Test-Presenter?  I've
checked in fixes that we've tested over the past couple of weeks.
There's a conference coming up in a month that we'd like to have these
fixes for.

Thanks,
Mark

Re: [Crucible-devel] [patch] basic.sh - make installcheck bug

[Bryce H. <br...@br...>]

Thanks, applied.  Yeah that looks like maybe it was an oversight.  I
*think* that the -s was added to get around some other issue, but I
don't remember.  But if anyone notices failures in the installcheck
stuff that seems related to hostnames, we might want to doublecheck this
again.

Bryce

On Thu, Mar 01, 2007 at 04:06:47PM -0800, Leann Ogasawara wrote:
> Hi Guys,
> 
> I noticed a small discrepancy between the basic.sh script (used when
> running 'make installcheck') and the sut script.  basic.sh sets $sut as
> follows:
> 
> 	sut=`hostname -s`
> 
> basic.sh adds $sut as a new sut:
> 
> 	sut add $sut
> 
> Then, in the sut script $driver is set:
> 
> 	driver=`hostname`
> 
> The values of $driver and $sut are compared to see if the sut being
> added is not the driver.  NFS mount point are then established if the
> sut and driver are not the same.  However, because sut is being set to
> `hostname -s` and driver to just `hostname` even if the driver and sut
> are the same machine, it might not appear that way because `hostname`
> and `hostname -s` may return different values.  This was the case I ran
> into which results in the 'make installcheck' failing.
> 
> The following patch fixes the issue for me.  It just a simple one liner.
> 
> Thanks,
> Leann
> 
> Index: crucible/t/installcheck/basic.sh
> ===================================================================
> --- crucible/t/installcheck/basic.sh	(revision 717)
> +++ crucible/t/installcheck/basic.sh	(working copy)
> @@ -38,7 +38,7 @@
>  
>  package_path="$PKGS_DIR/$pkg_name/$package"
>  plan_path="$PLANS_DIR/$plan"
> -sut=`hostname -s`
> +sut=`hostname`
>  
>  # Verify pre-requisites are all in place
>  ok "which queue_plan"            "Checking for 'queue_plan' script"

Re: [Crucible-devel] [patch] create nfs mount points on sut

[Bryce H. <br...@br...>]

Thanks, applied!

Bryce

On Thu, Mar 01, 2007 at 12:30:42PM -0800, Leann Ogasawara wrote:
> Hi All,
> 
> When using the 'sut' script to add a new sut, the directories for the
> nfs mount points weren't being created.  The patch below creates the
> directories accordingly.  I also did some whitespace cleanups (converted
> tabs to spaces) as that appears to be the preferred format.  The patch
> has been tested with 'sut add <machine>'.
> 
> Comments welcome.
> 
> Thanks,
> Leann
> 
> Index: crucible/scripts/sut
> ===================================================================
> --- crucible/scripts/sut	(revision 713)
> +++ crucible/scripts/sut	(working copy)
> @@ -125,6 +125,7 @@
>              return $retval
>          fi
>          nfs_regex=`echo $nfs_mountpoints | sed 's/\s/|/g'`
> +        nfs_mountpoints=`echo $nfs_mountpoints | sed 's/\s/ /g'`
>          nfs_fstab=
>          for m in $nfs_mountpoints; do
>              nfs_fstab="$nfs_fstab
> @@ -140,25 +141,27 @@
>      # TODO:  What if sut doesn't have nfs turned on?
>      #        Can we detect it?
>      if [ "x$sut_id" == "x$driver" ] || [ "x$sut_id" == "xlocalhost" ]; then
> -        crontab -l | grep -v sut_dispatcher >/tmp/crontab-new || exit 25
> -	echo "$cronjob" >>/tmp/crontab-new || exit 26
> +        crontab -l | grep -v sut_dispatcher >/tmp/crontab-new || exit 26
> +        echo "$cronjob" >>/tmp/crontab-new || exit 27
>          echo 'Installing sut_dispatcher cronjob...'
> -	crontab /tmp/crontab-new || exit 27
> -	sleep 1
> +        crontab /tmp/crontab-new || exit 28
> +        sleep 1
>      else
>        ssh $sut_id -t "echo 'Setting up config file...'; \
>          [ \"$cru_config\" ] && echo \"$cru_config\" > $config_file; \
>          cp /etc/fstab /etc/fstab.bak || exit 21; \
>          egrep -v \"$nfs_regex\" /etc/fstab.bak > /etc/fstab || exit 22; \
> -        echo 'Creating NFS mounts...'; \
> -        echo \"$nfs_fstab\" >> /etc/fstab || exit 23; \
> +        echo 'Creating NFS mount points...'; \
> +        mkdir -p $nfs_mountpoints || exit 23; \
> +        echo 'Updating /etc/fstab...'; \
> +        echo \"$nfs_fstab\" >> /etc/fstab || exit 24; \
>          echo 'Mounting...'; \
> -        mount -a -tnfs || exit 24; \
> -        crontab -l | grep -v sut_dispatcher >/tmp/crontab-new || exit 25; \
> -	echo \"$cronjob\" >>/tmp/crontab-new || exit 26; \
> +        mount -a -tnfs || exit 25; \
> +        crontab -l | grep -v sut_dispatcher >/tmp/crontab-new || exit 26; \
> +        echo \"$cronjob\" >>/tmp/crontab-new || exit 27; \
>          echo 'Installing sut_dispatcher cronjob...'; \
> -	crontab /tmp/crontab-new || exit 27; \
> -	sleep 1"
> +        crontab /tmp/crontab-new || exit 28; \
> +        sleep 1"
>      fi
>  
>      retval=$?

Re: [Crucible-devel] [patch for RFC] hooks for netconsole logging

[Bryce H. <br...@br...>]

On Fri, Mar 30, 2007 at 01:37:01PM -0700, Randy Dunlap wrote:
> 
> Comments?  can I commit all or part of this?
> (without some of the debug prints/echos)

Yup, this looks good.  If you've not committed already, please do so.  

Bryce
 
> Also attached is plans/reports/09-netconsole that I am using with this.
> 
> ---
> 
> Add hooks in dispatchers so that if netconlog=1 in /testing/etc/global.cfg,
> netcon_start_log is called when a new testrun begins running and
> netcon_kill_log is called when a testrun finishes running.
> This is not done if the SUT is defined as sut_local in global.cfg.
> 
> Change instances of base_dir/etc/global.cfg to ETC_DIR/global.cfg.
> 
> Change ar_dispatcher use of "oldest" and "newest" in echo messages from:
> oldest run id=1463, newest run id=1
> to
> newest run id=1463, oldest run id=1
> 
> Add 'sut_is_local' function in common.fns.  It uses etc/global.cfg::sut_locals
> to define the local SUT(s).  This is used along with etc/global.cfg::netconlog
> to determine if netcat should be run for a SUT.
> 
> ---
>  plans/commands/common.fns  |   24 ++++++++++++++++++++++++
>  usr/bin/ar_dispatcher      |   17 +++++++++++++----
>  usr/bin/plan_dispatcher    |    3 +--
>  usr/bin/report_dispatcher  |    3 +--
>  usr/bin/request_dispatcher |    3 +--
>  usr/bin/sut_dispatcher     |    3 +--
>  6 files changed, 41 insertions(+), 12 deletions(-)
> 
> --- cruc.orig/usr/bin/sut_dispatcher
> +++ cruc/usr/bin/sut_dispatcher
> @@ -34,7 +34,7 @@ base_dir=${BASE_DIR:-/testing}
>  . $base_dir/plans/commands/common.fns
>  
>  # source global var file
> -. $base_dir/etc/global.cfg || exit 1
> +. $ETC_DIR/global.cfg || exit 1
>  
>  sut_id=${SUT_ID:-`hostname --short`}
>  err_delay=${ERR_DELAY:-10}
> @@ -167,4 +167,3 @@ while :; do
>          finish_run $run_id
>      fi
>  done
> -
> --- cruc.orig/usr/bin/ar_dispatcher
> +++ cruc/usr/bin/ar_dispatcher
> @@ -14,7 +14,7 @@ base_dir=${BASE_DIR:-/testing}
>  . $base_dir/plans/commands/common.fns
>  
>  # source global var file
> -. $base_dir/etc/global.cfg || exit 1
> +. $ETC_DIR/global.cfg || exit 1
>  
>  # Define extra vars
>  err_delay=20
> @@ -39,20 +39,20 @@ if [ ! -d "$RUNS_DIR" ]; then
>      exit -1
>  fi
>  
> -# Find youngest run id
> +# Find oldest run id
>  ID=`(cd "$RUNS_DIR" && ls) | awk '{val=$0+0;
>                                         if (min == 0 && val>0)  { min=val; };
>                                         if (val < min && val!=0) { min=val; };
>                                        }
>                                        END{print min+0;}'`
> -# Find oldest run id
> +# Find newest run id
>  LAST_ID=`(cd "$RUNS_DIR" && ls) | awk '{val=$0+0;
>                                         if (max == 0 && val>0)  { max=val; };
>                                         if (val > max && val!=0) { max=val; };
>                                        }
>                                        END{print max+0;}'`
>  
> -dbgmsg "oldest run id=%d, newest run id=%d\n" $LAST_ID $ID
> +dbgmsg "newest run id=%d, oldest run id=%d\n" $LAST_ID $ID
>  
>  get_next_run_id() {
>      # Make sure we have a non-zero starting point
> @@ -197,6 +197,15 @@ while :; do
>              echo "Initiating test '$ID'"
>              touch "$RUNS_DIR/$ID/running"
>              rm -f "$RUNS_DIR/$ID/pending"
> +
> +	    sut_is_local "$suts"
> +	    locl=$?
> +	    if [ "$netconlog" == "1" ] && [ "$locl" == "0" ]; then
> +		echo "do netcon_start_log: $RUNS_DIR/$ID"
> +		netcon_start_log $RUNS_DIR/$ID
> +	    else
> +		echo "bah, netconlog==$netconlog or sut_is_local:$suts"
> +	    fi
>          fi
>  
>          ID=$(( ID + 1 ))
> --- cruc.orig/usr/bin/plan_dispatcher
> +++ cruc/usr/bin/plan_dispatcher
> @@ -39,7 +39,7 @@ base_dir=${BASE_DIR:-/testing}
>  . $base_dir/plans/commands/common.fns
>  
>  # source global var file
> -. $base_dir/etc/global.cfg || exit 1
> +. $ETC_DIR/global.cfg || exit 1
>  
>  # Define extra vars
>  tmp_dir=/tmp
> @@ -139,4 +139,3 @@ while :; do
>  	rm "$pkg"
>      fi    
>  done
> -
> --- cruc.orig/usr/bin/report_dispatcher
> +++ cruc/usr/bin/report_dispatcher
> @@ -15,7 +15,7 @@ base_dir=${BASE_DIR:-/testing}
>  . $base_dir/plans/commands/common.fns
>  
>  # source global var file
> -. $base_dir/etc/global.cfg || exit 1
> +. $ETC_DIR/global.cfg || exit 1
>  
>  # Define extra vars
>  report_dir=$PLANS_DIR/reports
> @@ -70,4 +70,3 @@ while :; do
>  
>      sleep $err_delay
>  done
> -
> --- cruc.orig/usr/bin/request_dispatcher
> +++ cruc/usr/bin/request_dispatcher
> @@ -14,7 +14,7 @@ base_dir=${BASE_DIR:-/testing}
>  . $base_dir/plans/commands/common.fns
>  
>  # source global var file
> -. $base_dir/etc/global.cfg || exit 1
> +. $ETC_DIR/global.cfg || exit 1
>  
>  # Define extra vars
>  test_dir=$PLANS_DIR/commands
> @@ -180,4 +180,3 @@ while :; do
>        rm "$plan"
>      fi
>  done
> -
> --- cruc.orig/plans/commands/common.fns
> +++ cruc/plans/commands/common.fns
> @@ -129,3 +129,27 @@ dbgmsg() {
>  		printf "$@" >&2
>  	fi
>  }
> +
> +# sut_is_local $suts
> +#
> +# Returns 1 if the SUT in $suts is listed in ETC_DIR/sut_locals
> +# else returns 0.
> +# Expects only one SUT listed in $suts.
> +#
> +sut_is_local () {
> +        local sutlist=$1
> +        local sutlocal
> +
> +	printf "sutlist:%s\n" $sutlist >&2
> +	printf "sutlocl:%s\n" $sut_locals >&2
> +
> +        for loc in $sut_locals ; do
> +                for sut in $sutlist ; do
> +                        if [ "$loc" == "$sut" ]; then
> +                                return 1
> +                        fi
> +                done
> +        done
> +
> +	return 0
> +}

[Crucible-devel] [patch for RFC] hooks for netconsole logging

[Randy D. <rd...@xe...>]

Comments?  can I commit all or part of this?
(without some of the debug prints/echos)

Also attached is plans/reports/09-netconsole that I am using with this.

---

Add hooks in dispatchers so that if netconlog=1 in /testing/etc/global.cfg,
netcon_start_log is called when a new testrun begins running and
netcon_kill_log is called when a testrun finishes running.
This is not done if the SUT is defined as sut_local in global.cfg.

Change instances of base_dir/etc/global.cfg to ETC_DIR/global.cfg.

Change ar_dispatcher use of "oldest" and "newest" in echo messages from:
oldest run id=1463, newest run id=1
to
newest run id=1463, oldest run id=1

Add 'sut_is_local' function in common.fns.  It uses etc/global.cfg::sut_locals
to define the local SUT(s).  This is used along with etc/global.cfg::netconlog
to determine if netcat should be run for a SUT.

---
 plans/commands/common.fns  |   24 ++++++++++++++++++++++++
 usr/bin/ar_dispatcher      |   17 +++++++++++++----
 usr/bin/plan_dispatcher    |    3 +--
 usr/bin/report_dispatcher  |    3 +--
 usr/bin/request_dispatcher |    3 +--
 usr/bin/sut_dispatcher     |    3 +--
 6 files changed, 41 insertions(+), 12 deletions(-)

--- cruc.orig/usr/bin/sut_dispatcher
+++ cruc/usr/bin/sut_dispatcher
@@ -34,7 +34,7 @@ base_dir=${BASE_DIR:-/testing}
 . $base_dir/plans/commands/common.fns
 
 # source global var file
-. $base_dir/etc/global.cfg || exit 1
+. $ETC_DIR/global.cfg || exit 1
 
 sut_id=${SUT_ID:-`hostname --short`}
 err_delay=${ERR_DELAY:-10}
@@ -167,4 +167,3 @@ while :; do
         finish_run $run_id
     fi
 done
-
--- cruc.orig/usr/bin/ar_dispatcher
+++ cruc/usr/bin/ar_dispatcher
@@ -14,7 +14,7 @@ base_dir=${BASE_DIR:-/testing}
 . $base_dir/plans/commands/common.fns
 
 # source global var file
-. $base_dir/etc/global.cfg || exit 1
+. $ETC_DIR/global.cfg || exit 1
 
 # Define extra vars
 err_delay=20
@@ -39,20 +39,20 @@ if [ ! -d "$RUNS_DIR" ]; then
     exit -1
 fi
 
-# Find youngest run id
+# Find oldest run id
 ID=`(cd "$RUNS_DIR" && ls) | awk '{val=$0+0;
                                        if (min == 0 && val>0)  { min=val; };
                                        if (val < min && val!=0) { min=val; };
                                       }
                                       END{print min+0;}'`
-# Find oldest run id
+# Find newest run id
 LAST_ID=`(cd "$RUNS_DIR" && ls) | awk '{val=$0+0;
                                        if (max == 0 && val>0)  { max=val; };
                                        if (val > max && val!=0) { max=val; };
                                       }
                                       END{print max+0;}'`
 
-dbgmsg "oldest run id=%d, newest run id=%d\n" $LAST_ID $ID
+dbgmsg "newest run id=%d, oldest run id=%d\n" $LAST_ID $ID
 
 get_next_run_id() {
     # Make sure we have a non-zero starting point
@@ -197,6 +197,15 @@ while :; do
             echo "Initiating test '$ID'"
             touch "$RUNS_DIR/$ID/running"
             rm -f "$RUNS_DIR/$ID/pending"
+
+	    sut_is_local "$suts"
+	    locl=$?
+	    if [ "$netconlog" == "1" ] && [ "$locl" == "0" ]; then
+		echo "do netcon_start_log: $RUNS_DIR/$ID"
+		netcon_start_log $RUNS_DIR/$ID
+	    else
+		echo "bah, netconlog==$netconlog or sut_is_local:$suts"
+	    fi
         fi
 
         ID=$(( ID + 1 ))
--- cruc.orig/usr/bin/plan_dispatcher
+++ cruc/usr/bin/plan_dispatcher
@@ -39,7 +39,7 @@ base_dir=${BASE_DIR:-/testing}
 . $base_dir/plans/commands/common.fns
 
 # source global var file
-. $base_dir/etc/global.cfg || exit 1
+. $ETC_DIR/global.cfg || exit 1
 
 # Define extra vars
 tmp_dir=/tmp
@@ -139,4 +139,3 @@ while :; do
 	rm "$pkg"
     fi    
 done
-
--- cruc.orig/usr/bin/report_dispatcher
+++ cruc/usr/bin/report_dispatcher
@@ -15,7 +15,7 @@ base_dir=${BASE_DIR:-/testing}
 . $base_dir/plans/commands/common.fns
 
 # source global var file
-. $base_dir/etc/global.cfg || exit 1
+. $ETC_DIR/global.cfg || exit 1
 
 # Define extra vars
 report_dir=$PLANS_DIR/reports
@@ -70,4 +70,3 @@ while :; do
 
     sleep $err_delay
 done
-
--- cruc.orig/usr/bin/request_dispatcher
+++ cruc/usr/bin/request_dispatcher
@@ -14,7 +14,7 @@ base_dir=${BASE_DIR:-/testing}
 . $base_dir/plans/commands/common.fns
 
 # source global var file
-. $base_dir/etc/global.cfg || exit 1
+. $ETC_DIR/global.cfg || exit 1
 
 # Define extra vars
 test_dir=$PLANS_DIR/commands
@@ -180,4 +180,3 @@ while :; do
       rm "$plan"
     fi
 done
-
--- cruc.orig/plans/commands/common.fns
+++ cruc/plans/commands/common.fns
@@ -129,3 +129,27 @@ dbgmsg() {
 		printf "$@" >&2
 	fi
 }
+
+# sut_is_local $suts
+#
+# Returns 1 if the SUT in $suts is listed in ETC_DIR/sut_locals
+# else returns 0.
+# Expects only one SUT listed in $suts.
+#
+sut_is_local () {
+        local sutlist=$1
+        local sutlocal
+
+	printf "sutlist:%s\n" $sutlist >&2
+	printf "sutlocl:%s\n" $sut_locals >&2
+
+        for loc in $sut_locals ; do
+                for sut in $sutlist ; do
+                        if [ "$loc" == "$sut" ]; then
+                                return 1
+                        fi
+                done
+        done
+
+	return 0
+}

[Crucible-devel] [patch] basic.sh - make installcheck bug

[Leann O. <log...@li...>]

Hi Guys,

I noticed a small discrepancy between the basic.sh script (used when
running 'make installcheck') and the sut script.  basic.sh sets $sut as
follows:

	sut=`hostname -s`

basic.sh adds $sut as a new sut:

	sut add $sut

Then, in the sut script $driver is set:

	driver=`hostname`

The values of $driver and $sut are compared to see if the sut being
added is not the driver.  NFS mount point are then established if the
sut and driver are not the same.  However, because sut is being set to
`hostname -s` and driver to just `hostname` even if the driver and sut
are the same machine, it might not appear that way because `hostname`
and `hostname -s` may return different values.  This was the case I ran
into which results in the 'make installcheck' failing.

The following patch fixes the issue for me.  It just a simple one liner.

Thanks,
Leann

Index: crucible/t/installcheck/basic.sh
===================================================================
--- crucible/t/installcheck/basic.sh	(revision 717)
+++ crucible/t/installcheck/basic.sh	(working copy)
@@ -38,7 +38,7 @@
 
 package_path="$PKGS_DIR/$pkg_name/$package"
 plan_path="$PLANS_DIR/$plan"
-sut=`hostname -s`
+sut=`hostname`
 
 # Verify pre-requisites are all in place
 ok "which queue_plan"            "Checking for 'queue_plan' script"

[Crucible-devel] [patch] create nfs mount points on sut

[Leann O. <log...@li...>]

Hi All,

When using the 'sut' script to add a new sut, the directories for the
nfs mount points weren't being created.  The patch below creates the
directories accordingly.  I also did some whitespace cleanups (converted
tabs to spaces) as that appears to be the preferred format.  The patch
has been tested with 'sut add <machine>'.

Comments welcome.

Thanks,
Leann

Index: crucible/scripts/sut
===================================================================
--- crucible/scripts/sut	(revision 713)
+++ crucible/scripts/sut	(working copy)
@@ -125,6 +125,7 @@
             return $retval
         fi
         nfs_regex=`echo $nfs_mountpoints | sed 's/\s/|/g'`
+        nfs_mountpoints=`echo $nfs_mountpoints | sed 's/\s/ /g'`
         nfs_fstab=
         for m in $nfs_mountpoints; do
             nfs_fstab="$nfs_fstab
@@ -140,25 +141,27 @@
     # TODO:  What if sut doesn't have nfs turned on?
     #        Can we detect it?
     if [ "x$sut_id" == "x$driver" ] || [ "x$sut_id" == "xlocalhost" ]; then
-        crontab -l | grep -v sut_dispatcher >/tmp/crontab-new || exit 25
-	echo "$cronjob" >>/tmp/crontab-new || exit 26
+        crontab -l | grep -v sut_dispatcher >/tmp/crontab-new || exit 26
+        echo "$cronjob" >>/tmp/crontab-new || exit 27
         echo 'Installing sut_dispatcher cronjob...'
-	crontab /tmp/crontab-new || exit 27
-	sleep 1
+        crontab /tmp/crontab-new || exit 28
+        sleep 1
     else
       ssh $sut_id -t "echo 'Setting up config file...'; \
         [ \"$cru_config\" ] && echo \"$cru_config\" > $config_file; \
         cp /etc/fstab /etc/fstab.bak || exit 21; \
         egrep -v \"$nfs_regex\" /etc/fstab.bak > /etc/fstab || exit 22; \
-        echo 'Creating NFS mounts...'; \
-        echo \"$nfs_fstab\" >> /etc/fstab || exit 23; \
+        echo 'Creating NFS mount points...'; \
+        mkdir -p $nfs_mountpoints || exit 23; \
+        echo 'Updating /etc/fstab...'; \
+        echo \"$nfs_fstab\" >> /etc/fstab || exit 24; \
         echo 'Mounting...'; \
-        mount -a -tnfs || exit 24; \
-        crontab -l | grep -v sut_dispatcher >/tmp/crontab-new || exit 25; \
-	echo \"$cronjob\" >>/tmp/crontab-new || exit 26; \
+        mount -a -tnfs || exit 25; \
+        crontab -l | grep -v sut_dispatcher >/tmp/crontab-new || exit 26; \
+        echo \"$cronjob\" >>/tmp/crontab-new || exit 27; \
         echo 'Installing sut_dispatcher cronjob...'; \
-	crontab /tmp/crontab-new || exit 27; \
-	sleep 1"
+        crontab /tmp/crontab-new || exit 28; \
+        sleep 1"
     fi
 
     retval=$?
 

Re: [Crucible-devel] need a Test-Presenter release

[<ma...@gm...>]

On 2/20/07, Bryce Harrington <br...@br...> wrote:
> On Sun, Feb 18, 2007 at 09:02:39PM -0800, ma...@gm... wrote:
> > On 2/12/07, Bryce Harrington <br...@br...> wrote:
> > > On Mon, Feb 12, 2007 at 04:48:04PM -0800, ma...@gm... wrote:
> > > > >No, just SourceForge.  I wasn't sure if it's mature enough for CPAN, but
> > > > >if you need it there, I guess there's no reason it can't be and I'd be
> > > > >happy to upload it.  Lemme know.
> > > >
> > > > Yeah, it'll be helpful if it's on cpan too.
> > >
> > > Okay, it's posted to cpan.org too.  Should be available within a few
> > > hours, as normal.
> >
> > Hi Bryce,
> >
> > I don't see it on cpan yet, wanted to double check to see if there's a problem.
>
> Hmm, that's very strange; I was sure I'd uploaded it, but yet there's no
> sign of it.  I've just now uploaded it again to CPAN, sorry for the
> delay.  Let's see if it takes this time.

No problem, thanks.

Mark