xCAT / Bugs / #4457 xcat for ubuntu 14.10 won't run driver postinstall compile on genimage

ralph bellofatto - 2014-12-04

attaching a copy of the genimage.log file

genimage.log

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Guang Cheng Li - 2014-12-05

assigned_to: zhao er tao

component: unknown --> ubuntu
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Guang Cheng Li - 2014-12-05

Er Tao,

We could compile and configure the InfiniBand drivers using postinstall scripts with genimage, so it should be possible to compile and configure drivers in postinstall scripts, could you work with Ralph on this issue? Thanks.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

after the genimage is run (and fails once, the problem can be seen by running the following command directly under chroot:

chroot /install/netboot/ubuntu14.10/ppc64el/compute/rootimg  dpkg --configure -D 777 nvidia-340

We did some further research by instrumenting the post install script and what it calls and managed to duplicate the problem with just the following command:

chroot /install/netboot/ubuntu14.10/ppc64el/compute/rootimg  dkms status -m nvidia-340 -v 340.50 -k 3.16.0-24-generic -a ppc64el

When we run this under chroot it produces:

root@tulgpu002:/opt/xcat/share/xcat/netboot/ubuntu# chroot /install/netboot/ubuntu14.10/ppc64el/compute/rootimg  dkms status -m nvidia-340 -v 340.50 -k 3.16.0-24-generic -a ppc64el
/usr/sbin/dkms: line 1907: /dev/fd/62: No such file or directory
/usr/sbin/dkms: line 1837: /dev/fd/62: No such file or directory
root@tulgpu002:/opt/xcat/share/xcat/netboot/ubuntu# echo $?
1

when run natively this produces:

root@tulgpu002:/opt/xcat/share/xcat/netboot/ubuntu#   dkms status -m nvidia-340 -v 340.50 -k 3.16.0-24-generic -a ppc64el ; echo $?
nvidia-340, 340.50, 3.16.0-24-generic, ppc64el: installed
0

We instrumented the dkms code and found that it was attempting to use the following:

# Check to see if modules meeting the passed parameters are weak-installed.
# This function's calling convention is different from the usual DKMS status
# checking functions -- the kernel version we usually have is the one we are currently
# running on, not necessarily the one we compiled the module for.
module_status_weak() {
    # $1 = module, $2 = module version, $3 = kernel version weak installed to,
    # $4 = kernel arch, $5 = kernel version built for
    [[ $weak_modules ]] || return 1
    local weak_ko mod installed_ko f ret=1 oifs=$IFS
    local -a already_found
    for weak_ko in "$install_tree/"*/weak-updates/*; do
    [[ -e $weak_ko ]] || continue
    [[ -L $weak_ko ]] && installed_ko="$(readlink -f "$weak_ko")" || continue
    IFS=/ read m v k a < <(IFS=$oifs find_module_from_ko "$weak_ko") || continue
    kern=${weak_ko#$install_tree/}
    kern=${kern%/weak-updates/*}
    [[ $m = ${1:-*} && $v = ${2:-*} && $k = ${5:-*} && \
        $a = ${4:-*} && $kern = ${3:-*} ]] || \
        continue
    ret=0
    for f in "${already_found[@]}"; do
        [[ $f = $m/$v/$kern/$a/$k ]] && continue 2
    done
    already_found[${#already_found[@]}]="$m/$v/$kern/$a/$k"
    echo "installed-weak $m/$v/$kern/$a/$k"
    done
    return $ret
}

# Print the requested status lines for weak-installed modules.
do_status_weak()
{
    local mvka m v k a kern status
    while read status mvka; do
    IFS=/ read m v k a kern <<< "$mvka"
    echo "$m, $v, $k, $a: installed-weak from $kern"
    done < <(module_status_weak "$@")
}

We suspected the IO redirection of getting the output of a funciton is not compatable with running under chroot.

To test this theory we created a testme.sh script as follows:

#!/bin/bash


module_status_weak() {
    [[ $weak_modules ]] || return 1
    local weak_ko mod installed_ko f ret=1 oifs=$IFS
    return $ret
}

do_status_weak()
{
    local mvka m v k a kern status
    while read status mvka; do
    IFS=/ read m v k a kern <<< "$mvka"
    echo "$m, $v, $k, $a: installed-weak from $kern"
    done < <(module_status_weak "$@")
}

show_status()
{
    do_status_weak "$module" "$module_version" "${kernelver[$j]}" "${arch[$j]}"
}

action=status

weak_modules='';


for action_to_run in $action; do
    case "$action_to_run" in
    status)       show_status;;
    '') error $"No action was specified."
    ;;
    *)  error $"Unknown action specified: $action_to_run"
    ;;
    esac
done

Running this natively produced:

root@tulgpu002:/opt/xcat/share/xcat/netboot/ubuntu# /install/netboot/ubuntu14.10/ppc64el/compute/rootimg/testme.sh;
0

And running it under chroot produced:

root@tulgpu002:/opt/xcat/share/xcat/netboot/ubuntu# chroot /install/netboot/ubuntu14.10/ppc64el/compute/rootimg ./t
estme.sh; echo $?
./testme.sh: line 11: /dev/fd/62: No such file or directory
1

So it would appear that the technique used by dkms to get the status by redirecting the stdout
of a function does not work in ubuntu when running under chroot.

So now that we know that, how does one install drivers that use dkms as part of their post install process?

zhao er tao - 2014-12-10

Milestone: 2.9 --> 2.9.1
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- ralph bellofatto - 2014-12-10
  
  I found the answer to this problem.
  
  To run postinstall steps that will install and run init scripts on the
  system, one also needs to mount a /proc and /sys file system.
  
  We created a wrapper script for all our post-install scripts which allows
  the cuda drivers (and other drivers doing similar work with dkms to work
  correctly.
  
  #!/bin/sh #-- this script is run after all packages from $profile.pkglist are installed #-- #-- it gets these arguments: #-- #-- $1 = install root (chroot directory for profile) #-- $2 = OS version #-- $3 = architecture #-- $4 = profile name #-- $5 = work dir (where genimage is located) #-- #-- set -x set -e thisdir=`dirname $0` thisscript=`basename $0` installroot=$1 osver=$2 arch=$3 profile=$4 workdir=$5 proc_dir= sys_dir= # exit trap to undo any mounts done earlier function finish { set +e; [ -n $proc_dir ] && umount $proc_dir; [ -n $sys_dir ] && umount $sys_dir; } trap finish EXIT # # make sure proc and sys are available to anything running under chroot. # mount -o bind /sys $installroot/sys && sys_dir=$installroot/sys mount -o bind /proc $installroot/proc && proc_dir=$installroot/proc for s in $(find $thisdir/postinstall.d -type f -executable | sort); do echo "executing $s" rc=0; $s $installroot $osver $arch $profile $workdir || rc=$? if [[ $rc != 0 ]]; then echo "$s FAILED"; exit 1; fi; done
  
  Ralph Bellofatto
  IBM TJ Watson Research
  1-914-945-3321
  ralphbel@us.ibm.com
  
  From: "zhao er tao" zhaoertao@users.sf.net
  To: "[xcat:bugs] " 4457@bugs.xcat.p.re.sf.net
  Date: 12/10/2014 03:42 AM
  Subject: [xcat:bugs] #4457 xcat for ubuntu 14.10 won't run driver
  postinstall compile on genimage
  
  Milestone: 2.9 --> 2.9.1 Comment:
  
  After some investigation, I found this issue "/dev/fd/62: No such file or
  directory" is very difficult to be fixed in postinstall script of genimage.
  There are 2 options I can think to work around this issue:
  
  use postscripts
  
  install the cuda onto a diskfull node, and then merge all the files,
  directories, drivers, scripts to the rootimg created by genimage.
  
  We will need to work with cuda team on how to deal with this issue. So I
  move this defect to 2.9.1.
  
  [bugs:#4457] xcat for ubuntu 14.10 won't run driver postinstall compile on
  genimage
  
  Status: open
  Milestone: 2.9.1
  Created: Thu Dec 04, 2014 10:50 PM UTC by ralph bellofatto
  Last Updated: Tue Dec 09, 2014 02:57 PM UTC
  Owner: zhao er tao
  
  xcat for ubuntu 14.10 won't run driver postinstall compile on genimage
  
  I'm attempting to install nvidia drivers on to network boot.
  
  I am doing this with a post install script that looks like this:
  
  set -e
  
  thisdir=dirname $0
  thisscript=basename $0
  
  installroot=$1
  osver=$2
  arch=$3
  profile=$4
  workdir=$5
  
  make sure the debian front end does not produce this error:
  
  debconf: (Can't locate Debconf/FrontEnd/Newt.pm in @INC
  
  try this from: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=439763
  
  export DEBIAN_FRONTEND=noninteractive
  unset DEBIAN_HAS_FRONTEND
  unset DEBCONF_REDIR
  unset DEBCONF_OLD_FD_BASE
  unset ARCH
  
  chroot $installroot \
  apt-get -q -y --force-yes -o Dpkg::Options::="--force-confold" install
  cuda
  
  when i run genimage on the compute node I get:
  
  Done.
  Loading new nvidia-340-340.50 DKMS files...
  First Installation: checking all kernels...
  Building only for 3.16.0-24-generic
  Building for architecture ppc64el
  /usr/sbin/dkms: line 1902: /dev/fd/62: No such file or directory
  /usr/sbin/dkms: line 1833: /dev/fd/62: No such file or directory
  dpkg: error processing package nvidia-340 (--configure):
  subprocess installed post-installation script returned error exit status 1
  dpkg: dependency problems prevent configuration of nvidia-340-uvm:
  nvidia-340-uvm depends on nvidia-340 (>= 340.50); however:
  Package nvidia-340 is not configured yet.
  
  Errors were encountered while processing:
  nvidia-340
  nvidia-340-uvm
  nvidia-340-dev
  libcuda1-340
  cuda-drivers
  cuda-runtime-5-5-power8
  cuda-5-5-power8
  cuda
  E: Sub-process /usr/bin/dpkg returned an error code (1)
  postinstall
  script /install/postscripts/custom/tulgpu-0000-netboot-compute/compute.postinstall
  failed
  
  our osimage looks like this:
  
  tulgpu-0000-netboot-compute:
  objtype=osimage
  exlist=/opt/xcat/share/xcat/netboot/ubuntu/compute.exlist
  imagetype=linux
  
  kerneldir=/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
  kernelver=3.16.0-24-generic
  osarch=ppc64el
  osname=Linux
  osvers=ubuntu14.10
  
  otherpkgdir=http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
  utopic main restricted multiverse universe,
  http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
  utopic-backports main restricted multiverse universe,
  http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
  utopic-security main restricted multiverse universe,
  http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
  utopic-updates main restricted multiverse universe,
  http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu
  trusty at7.1,
  http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu
  trusty at8.0,
  http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/developer.download.nvidia.com/compute/cuda/repos/ubuntu!
  
  1410/ppc64el /
  
  otherpkglist=/install/custom/tulgpu-0000-netboot-compute/ubuntu/ppc64el/compute.otherpkg.pkglist
  
  pkgdir=/install/ubuntu14.10/ppc64el pkglist=/opt/xcat/share/xcat/netboot/ubuntu/compute.pkglist
  
  postinstall=/install/postscripts/custom/tulgpu-0000-netboot-compute/compute.postinstall
  
  postscripts=custom/tulgpu-0000-netboot-compute/compute.postscript profile=compute provmethod=netboot rootimgdir=/install/netboot/ubuntu14.10/ppc64el/compute
  
  synclists=/install/custom/tulgpu-0000-netboot-compute/ubuntu/ppc64el/compute.synclist
  
  The actual genimage command to get this far was:
  cd /opt/xcat/share/xcat/netboot/ubuntu; ./genimage -a ppc64el -o
  ubuntu14.10 -p compute -k 3.16.0-24-generic
  --kerneldir /install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
  --srcdir /install/ubuntu14.10/ppc64el
  --pkglist /opt/xcat/share/xcat/netboot/ubuntu/compute.pkglist --otherpkgdir
  "
  http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
  utopic main restricted multiverse universe,
  http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
  utopic-backports main restricted multiverse universe,
  http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
  utopic-security main restricted multiverse universe,
  http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
  utopic-updates main restricted multiverse universe,
  http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu
  trusty at7.1,http://10.0.0.1/install/mirrors/ubuntu1!
  4.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu trusty at8.0,
  http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/developer.download.nvidia.com/compute/cuda/repos/ubuntu1410/ppc64el
  /"
  --otherpkglist /install/custom/tulgpu-0000-netboot-compute/ubuntu/ppc64el/compute.otherpkg.pkglist
  
  --postinstall /install/postscripts/custom/tulgpu-0000-netboot-compute/compute.postinstall
  --rootimgdir /install/netboot/ubuntu14.10/ppc64el/compute
  tulgpu-0000-netboot-compute
  
  I have added linux-source and linux-headers-generic to the otherpkgs list
  and have verified that they got installed, we see this in the genimage.log
  file:
  
  Setting up linux-headers-generic (3.16.0.24.25) ...
  Setting up linux-source-3.16.0 (3.16.0-24.32) ...
  Setting up linux-source (3.16.0.24.25) ...
  
  Is this a generic problem witn drivers that have to compile themselves from
  source via dkms?
  
  What is the "proper" way to install drivers that have to compile from
  source on network boot images.
  
  Sent from sourceforge.net because you indicated interest in
  https://sourceforge.net/p/xcat/bugs/4457/
  
  To unsubscribe from further messages, please visit
  https://sourceforge.net/auth/subscriptions/
  
  Related
  
  Bugs: ~~#4457~~
  
  alternate
  
  graycol.gif
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
  - zhao er tao - 2014-12-11
    
    Hi, ralph, with your fix, do you get more issue after the CN is running with that diskless image? If not, we can accept your fix for cude issue.
    Formally, for the pkg installing that need to interact with kernel, we suggest to use postscript which will run after the diskless image kernel is running. The only concern is the installing process may take a long time to complete. So, do you have any idea how log will it cost to install cuda driver?
    
    If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
    - ralph bellofatto - 2014-12-11
      
      It takes about 10 minutes to install all of cuda. Given the boot time
      restrictions of the CORAL project, adding 10 extra minutes to boot the
      system is not an acceptable solution.
      
      This package install is not interacting with the kernel but interacting
      with dkms which is a script that handles re-compile and configuration of
      drivers. Specifically, it has a problem with the type of io redirection
      that dkms does from a function to a while loop. This causes an interaction
      between the script and the /dev/fd directory, which is a link to
      the /proc/self/fd directory.
      
      Any dpkg post install function that does anything similar will have the
      same problem.
      
      We have other problems with running the disk-less image that we built here,
      unrelated to building the cuda drivers. (
      http://sourceforge.net/p/xcat/bugs/4466).
      This specific problem does not appear to have anything to do with the cuda
      drivers or postinstall scripts in general, since when I remove the post
      install scripts, we still don't get a bootable image. We suspect something
      to do with the kernelver and kerneldir parameters in the osimage record at
      this point.
      
      Ralph Bellofatto
      IBM TJ Watson Research
      1-914-945-3321
      ralphbel@us.ibm.com
      
      From: "zhao er tao" zhaoertao@users.sf.net
      To: "[xcat:bugs] " 4457@bugs.xcat.p.re.sf.net
      Date: 12/10/2014 10:21 PM
      Subject: [xcat:bugs] Re: #4457 xcat for ubuntu 14.10 won't run driver
      postinstall compile on genimage
      
      Hi, ralph, with your fix, do you get more issue after the CN is running
      with that diskless image? If not, we can accept your fix for cude issue.
      Formally, for the pkg installing that need to interact with kernel, we
      suggest to use postscript which will run after the diskless image kernel is
      running. The only concern is the installing process may take a long time to
      complete. So, do you have any idea how log will it cost to install cuda
      driver?
      
      [bugs:#4457] xcat for ubuntu 14.10 won't run driver postinstall compile on
      genimage
      
      Status: open
      Milestone: 2.9.1
      Created: Thu Dec 04, 2014 10:50 PM UTC by ralph bellofatto
      Last Updated: Wed Dec 10, 2014 08:41 AM UTC
      Owner: zhao er tao
      
      xcat for ubuntu 14.10 won't run driver postinstall compile on genimage
      
      I'm attempting to install nvidia drivers on to network boot.
      
      I am doing this with a post install script that looks like this:
      
      set -e
      
      thisdir=dirname $0
      thisscript=basename $0
      
      installroot=$1
      osver=$2
      arch=$3
      profile=$4
      workdir=$5
      
      make sure the debian front end does not produce this error:
      
      debconf: (Can't locate Debconf/FrontEnd/Newt.pm in @INC
      
      try this from: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=439763
      
      export DEBIAN_FRONTEND=noninteractive
      unset DEBIAN_HAS_FRONTEND
      unset DEBCONF_REDIR
      unset DEBCONF_OLD_FD_BASE
      unset ARCH
      
      chroot $installroot \
      apt-get -q -y --force-yes -o Dpkg::Options::="--force-confold" install
      cuda
      
      when i run genimage on the compute node I get:
      
      Done.
      Loading new nvidia-340-340.50 DKMS files...
      First Installation: checking all kernels...
      Building only for 3.16.0-24-generic
      Building for architecture ppc64el
      /usr/sbin/dkms: line 1902: /dev/fd/62: No such file or directory
      /usr/sbin/dkms: line 1833: /dev/fd/62: No such file or directory
      dpkg: error processing package nvidia-340 (--configure):
      subprocess installed post-installation script returned error exit status 1
      dpkg: dependency problems prevent configuration of nvidia-340-uvm:
      nvidia-340-uvm depends on nvidia-340 (>= 340.50); however:
      Package nvidia-340 is not configured yet.
      
      Errors were encountered while processing:
      nvidia-340
      nvidia-340-uvm
      nvidia-340-dev
      libcuda1-340
      cuda-drivers
      cuda-runtime-5-5-power8
      cuda-5-5-power8
      cuda
      E: Sub-process /usr/bin/dpkg returned an error code (1)
      postinstall
      script /install/postscripts/custom/tulgpu-0000-netboot-compute/compute.postinstall
      failed
      
      our osimage looks like this:
      
      tulgpu-0000-netboot-compute:
      objtype=osimage
      exlist=/opt/xcat/share/xcat/netboot/ubuntu/compute.exlist
      imagetype=linux
      
      kerneldir=/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
      kernelver=3.16.0-24-generic
      osarch=ppc64el
      osname=Linux
      osvers=ubuntu14.10
      
      otherpkgdir=http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
      utopic main restricted multiverse universe,
      http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
      utopic-backports main restricted multiverse universe,
      http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
      utopic-security main restricted multiverse universe,
      http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
      utopic-updates main restricted multiverse universe,
      http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu
      trusty at7.1,
      http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu
      trusty at8.0,
      http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/developer.download.nvidia.com/compute/cuda/repos/ubuntu!
      
      1410/ppc64el /
      
      otherpkglist=/install/custom/tulgpu-0000-netboot-compute/ubuntu/ppc64el/compute.otherpkg.pkglist
      
      pkgdir=/install/ubuntu14.10/ppc64el pkglist=/opt/xcat/share/xcat/netboot/ubuntu/compute.pkglist
      
      postinstall=/install/postscripts/custom/tulgpu-0000-netboot-compute/compute.postinstall
      
      postscripts=custom/tulgpu-0000-netboot-compute/compute.postscript profile=compute provmethod=netboot rootimgdir=/install/netboot/ubuntu14.10/ppc64el/compute
      
      synclists=/install/custom/tulgpu-0000-netboot-compute/ubuntu/ppc64el/compute.synclist
      
      The actual genimage command to get this far was:
      cd /opt/xcat/share/xcat/netboot/ubuntu; ./genimage -a ppc64el -o
      ubuntu14.10 -p compute -k 3.16.0-24-generic
      --kerneldir /install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
      --srcdir /install/ubuntu14.10/ppc64el
      --pkglist /opt/xcat/share/xcat/netboot/ubuntu/compute.pkglist --otherpkgdir
      "
      http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
      utopic main restricted multiverse universe,
      http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
      utopic-backports main restricted multiverse universe,
      http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
      utopic-security main restricted multiverse universe,
      http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
      utopic-updates main restricted multiverse universe,
      http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu
      trusty at7.1,http://10.0.0.1/install/mirrors/ubuntu1!
      4.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu trusty at8.0,
      http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/developer.download.nvidia.com/compute/cuda/repos/ubuntu1410/ppc64el
      /"
      --otherpkglist /install/custom/tulgpu-0000-netboot-compute/ubuntu/ppc64el/compute.otherpkg.pkglist
      
      --postinstall /install/postscripts/custom/tulgpu-0000-netboot-compute/compute.postinstall
      --rootimgdir /install/netboot/ubuntu14.10/ppc64el/compute
      tulgpu-0000-netboot-compute
      
      I have added linux-source and linux-headers-generic to the otherpkgs list
      and have verified that they got installed, we see this in the genimage.log
      file:
      
      Setting up linux-headers-generic (3.16.0.24.25) ...
      Setting up linux-source-3.16.0 (3.16.0-24.32) ...
      Setting up linux-source (3.16.0.24.25) ...
      
      Is this a generic problem witn drivers that have to compile themselves from
      source via dkms?
      
      What is the "proper" way to install drivers that have to compile from
      source on network boot images.
      
      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/xcat/bugs/4457/
      
      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/
      
      Related
      
      Bugs: ~~#4457~~
      
      alternate
      
      graycol.gif
      
      If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
      - ralph bellofatto - 2014-12-11
        
        Given what was found already, the solution to this may be just some
        application notes about how to run various things in a post install script.
        
        Anything that requires chroot dpkg --install, or apt-get, should probably
        be accompanied by the mount and unmount that i put in our wrapper script.
        
        Ralph Bellofatto
        IBM TJ Watson Research
        1-914-945-3321
        ralphbel@us.ibm.com
        
        From: "ralph bellofatto" ralphbel@users.sf.net
        To: "[xcat:bugs] " 4457@bugs.xcat.p.re.sf.net
        Date: 12/11/2014 08:35 AM
        Subject: [xcat:bugs] Re: #4457 xcat for ubuntu 14.10 won't run driver
        postinstall compile on genimage
        
        It takes about 10 minutes to install all of cuda. Given the boot time
        restrictions of the CORAL project, adding 10 extra minutes to boot the
        system is not an acceptable solution.
        
        This package install is not interacting with the kernel but interacting
        with dkms which is a script that handles re-compile and configuration of
        drivers. Specifically, it has a problem with the type of io redirection
        that dkms does from a function to a while loop. This causes an interaction
        between the script and the /dev/fd directory, which is a link to
        the /proc/self/fd directory.
        
        Any dpkg post install function that does anything similar will have the
        same problem.
        
        We have other problems with running the disk-less image that we built here,
        unrelated to building the cuda drivers. (
        http://sourceforge.net/p/xcat/bugs/4466).
        This specific problem does not appear to have anything to do with the cuda
        drivers or postinstall scripts in general, since when I remove the post
        install scripts, we still don't get a bootable image. We suspect something
        to do with the kernelver and kerneldir parameters in the osimage record at
        this point.
        
        Ralph Bellofatto
        IBM TJ Watson Research
        1-914-945-3321
        ralphbel@us.ibm.com
        
        From: "zhao er tao" zhaoertao@users.sf.net
        To: "[xcat:bugs] " 4457@bugs.xcat.p.re.sf.net
        Date: 12/10/2014 10:21 PM
        Subject: [xcat:bugs] Re: #4457 xcat for ubuntu 14.10 won't run driver
        postinstall compile on genimage
        
        Hi, ralph, with your fix, do you get more issue after the CN is running
        with that diskless image? If not, we can accept your fix for cude issue.
        Formally, for the pkg installing that need to interact with kernel, we
        suggest to use postscript which will run after the diskless image kernel is
        running. The only concern is the installing process may take a long time to
        complete. So, do you have any idea how log will it cost to install cuda
        driver?
        
        [bugs:#4457] xcat for ubuntu 14.10 won't run driver postinstall compile on
        genimage
        
        Status: open
        Milestone: 2.9.1
        Created: Thu Dec 04, 2014 10:50 PM UTC by ralph bellofatto
        Last Updated: Wed Dec 10, 2014 08:41 AM UTC
        Owner: zhao er tao
        
        xcat for ubuntu 14.10 won't run driver postinstall compile on genimage
        
        I'm attempting to install nvidia drivers on to network boot.
        
        I am doing this with a post install script that looks like this:
        
        set -e
        
        thisdir=dirname $0
        thisscript=basename $0
        
        installroot=$1
        osver=$2
        arch=$3
        profile=$4
        workdir=$5
        
        make sure the debian front end does not produce this error:
        debconf: (Can't locate Debconf/FrontEnd/Newt.pm in @INC
        try this from: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=439763
        
        export DEBIAN_FRONTEND=noninteractive
        unset DEBIAN_HAS_FRONTEND
        unset DEBCONF_REDIR
        unset DEBCONF_OLD_FD_BASE
        unset ARCH
        
        chroot $installroot \ apt-get -q -y --force-yes -o
        Dpkg::Options::="--force-confold" install
        cuda
        
        when i run genimage on the compute node I get:
        
        Done.
        Loading new nvidia-340-340.50 DKMS files...
        First Installation: checking all kernels...
        Building only for 3.16.0-24-generic
        Building for architecture ppc64el
        /usr/sbin/dkms: line 1902: /dev/fd/62: No such file or directory
        /usr/sbin/dkms: line 1833: /dev/fd/62: No such file or directory
        dpkg: error processing package nvidia-340 (--configure):
        subprocess installed post-installation script returned error exit status 1
        dpkg: dependency problems prevent configuration of nvidia-340-uvm:
        nvidia-340-uvm depends on nvidia-340 (>= 340.50); however:
        Package nvidia-340 is not configured yet.
        
        Errors were encountered while processing:
        nvidia-340
        nvidia-340-uvm
        nvidia-340-dev
        libcuda1-340
        cuda-drivers
        cuda-runtime-5-5-power8
        cuda-5-5-power8
        cuda
        E: Sub-process /usr/bin/dpkg returned an error code (1)
        postinstall
        script /install/postscripts/custom/tulgpu-0000-netboot-compute/compute.postinstall
        
        failed
        
        our osimage looks like this:
        
        tulgpu-0000-netboot-compute:
        objtype=osimage
        exlist=/opt/xcat/share/xcat/netboot/ubuntu/compute.exlist
        imagetype=linux
        
        kerneldir=/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
        kernelver=3.16.0-24-generic
        osarch=ppc64el
        osname=Linux
        osvers=ubuntu14.10
        
        otherpkgdir=http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
        
        utopic main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
        
        utopic-backports main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
        
        utopic-security main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
        
        utopic-updates main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu
        
        trusty at7.1,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu
        
        trusty at8.0,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/developer.download.nvidia.com/compute/cuda/repos/ubuntu!
        
        1410/ppc64el /
        
        otherpkglist=/install/custom/tulgpu-0000-netboot-compute/ubuntu/ppc64el/compute.otherpkg.pkglist
        
        pkgdir=/install/ubuntu14.10/ppc64el
        pkglist=/opt/xcat/share/xcat/netboot/ubuntu/compute.pkglist
        
        postinstall=/install/postscripts/custom/tulgpu-0000-netboot-compute/compute.postinstall
        
        postscripts=custom/tulgpu-0000-netboot-compute/compute.postscript
        profile=compute
        provmethod=netboot
        rootimgdir=/install/netboot/ubuntu14.10/ppc64el/compute
        
        synclists=/install/custom/tulgpu-0000-netboot-compute/ubuntu/ppc64el/compute.synclist
        
        The actual genimage command to get this far was:
        cd /opt/xcat/share/xcat/netboot/ubuntu; ./genimage -a ppc64el -o
        ubuntu14.10 -p compute -k 3.16.0-24-generic
        --kerneldir /install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
        
        --srcdir /install/ubuntu14.10/ppc64el
        --pkglist /opt/xcat/share/xcat/netboot/ubuntu/compute.pkglist --otherpkgdir
        "
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
        
        utopic main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
        
        utopic-backports main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
        
        utopic-security main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
        
        utopic-updates main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu
        
        trusty at7.1,http://10.0.0.1/install/mirrors/ubuntu1!
        4.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu trusty at8.0,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/developer.download.nvidia.com/compute/cuda/repos/ubuntu1410/ppc64el
        
        /"
        --otherpkglist /install/custom/tulgpu-0000-netboot-compute/ubuntu/ppc64el/compute.otherpkg.pkglist
        
        --postinstall /install/postscripts/custom/tulgpu-0000-netboot-compute/compute.postinstall
        
        --rootimgdir /install/netboot/ubuntu14.10/ppc64el/compute
        tulgpu-0000-netboot-compute
        
        I have added linux-source and linux-headers-generic to the otherpkgs list
        and have verified that they got installed, we see this in the genimage.log
        file:
        
        Setting up linux-headers-generic (3.16.0.24.25) ...
        Setting up linux-source-3.16.0 (3.16.0-24.32) ...
        Setting up linux-source (3.16.0.24.25) ...
        
        Is this a generic problem witn drivers that have to compile themselves from
        source via dkms?
        
        What is the "proper" way to install drivers that have to compile from
        source on network boot images.
        
        Sent from sourceforge.net because you indicated interest in
        https://sourceforge.net/p/xcat/bugs/4457/
        
        To unsubscribe from further messages, please visit
        https://sourceforge.net/auth/subscriptions/
        
        [bugs:#4457] xcat for ubuntu 14.10 won't run driver postinstall compile on
        genimage
        
        Status: open
        Milestone: 2.9.1
        Created: Thu Dec 04, 2014 10:50 PM UTC by ralph bellofatto
        Last Updated: Wed Dec 10, 2014 08:41 AM UTC
        Owner: zhao er tao
        
        xcat for ubuntu 14.10 won't run driver postinstall compile on genimage
        
        I'm attempting to install nvidia drivers on to network boot.
        
        I am doing this with a post install script that looks like this:
        
        set -e
        
        thisdir=dirname $0
        thisscript=basename $0
        
        installroot=$1
        osver=$2
        arch=$3
        profile=$4
        workdir=$5
        
        make sure the debian front end does not produce this error:
        
        debconf: (Can't locate Debconf/FrontEnd/Newt.pm in @INC
        
        try this from: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=439763
        
        export DEBIAN_FRONTEND=noninteractive
        unset DEBIAN_HAS_FRONTEND
        unset DEBCONF_REDIR
        unset DEBCONF_OLD_FD_BASE
        unset ARCH
        
        chroot $installroot \
        apt-get -q -y --force-yes -o Dpkg::Options::="--force-confold" install
        cuda
        
        when i run genimage on the compute node I get:
        
        Done.
        Loading new nvidia-340-340.50 DKMS files...
        First Installation: checking all kernels...
        Building only for 3.16.0-24-generic
        Building for architecture ppc64el
        /usr/sbin/dkms: line 1902: /dev/fd/62: No such file or directory
        /usr/sbin/dkms: line 1833: /dev/fd/62: No such file or directory
        dpkg: error processing package nvidia-340 (--configure):
        subprocess installed post-installation script returned error exit status 1
        dpkg: dependency problems prevent configuration of nvidia-340-uvm:
        nvidia-340-uvm depends on nvidia-340 (>= 340.50); however:
        Package nvidia-340 is not configured yet.
        
        Errors were encountered while processing:
        nvidia-340
        nvidia-340-uvm
        nvidia-340-dev
        libcuda1-340
        cuda-drivers
        cuda-runtime-5-5-power8
        cuda-5-5-power8
        cuda
        E: Sub-process /usr/bin/dpkg returned an error code (1)
        postinstall
        script /install/postscripts/custom/tulgpu-0000-netboot-compute/compute.postinstall
        failed
        
        our osimage looks like this:
        
        tulgpu-0000-netboot-compute:
        objtype=osimage
        exlist=/opt/xcat/share/xcat/netboot/ubuntu/compute.exlist
        imagetype=linux
        
        kerneldir=/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
        kernelver=3.16.0-24-generic
        osarch=ppc64el
        osname=Linux
        osvers=ubuntu14.10
        
        otherpkgdir=http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
        utopic main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
        utopic-backports main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
        utopic-security main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
        utopic-updates main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu
        trusty at7.1,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu
        trusty at8.0,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/developer.download.nvidia.com/compute/cuda/repos/ubuntu!
        
        1410/ppc64el /
        
        otherpkglist=/install/custom/tulgpu-0000-netboot-compute/ubuntu/ppc64el/compute.otherpkg.pkglist
        
        pkgdir=/install/ubuntu14.10/ppc64el pkglist=/opt/xcat/share/xcat/netboot/ubuntu/compute.pkglist
        
        postinstall=/install/postscripts/custom/tulgpu-0000-netboot-compute/compute.postinstall
        
        postscripts=custom/tulgpu-0000-netboot-compute/compute.postscript profile=compute provmethod=netboot rootimgdir=/install/netboot/ubuntu14.10/ppc64el/compute
        
        synclists=/install/custom/tulgpu-0000-netboot-compute/ubuntu/ppc64el/compute.synclist
        
        The actual genimage command to get this far was:
        cd /opt/xcat/share/xcat/netboot/ubuntu; ./genimage -a ppc64el -o
        ubuntu14.10 -p compute -k 3.16.0-24-generic
        --kerneldir /install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
        --srcdir /install/ubuntu14.10/ppc64el
        --pkglist /opt/xcat/share/xcat/netboot/ubuntu/compute.pkglist --otherpkgdir
        "
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
        utopic main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports
        utopic-backports main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
        utopic-security main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ports.ubuntu.com/ubuntu-ports/
        utopic-updates main restricted multiverse universe,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu
        trusty at7.1,http://10.0.0.1/install/mirrors/ubuntu1!
        4.10/mirror/ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu trusty at8.0,
        http://10.0.0.1/install/mirrors/ubuntu14.10/mirror/developer.download.nvidia.com/compute/cuda/repos/ubuntu1410/ppc64el
        /"
        --otherpkglist /install/custom/tulgpu-0000-netboot-compute/ubuntu/ppc64el/compute.otherpkg.pkglist
        
        --postinstall /install/postscripts/custom/tulgpu-0000-netboot-compute/compute.postinstall
        --rootimgdir /install/netboot/ubuntu14.10/ppc64el/compute
        tulgpu-0000-netboot-compute
        
        I have added linux-source and linux-headers-generic to the otherpkgs list
        and have verified that they got installed, we see this in the genimage.log
        file:
        
        Setting up linux-headers-generic (3.16.0.24.25) ...
        Setting up linux-source-3.16.0 (3.16.0-24.32) ...
        Setting up linux-source (3.16.0.24.25) ...
        
        Is this a generic problem witn drivers that have to compile themselves from
        source via dkms?
        
        What is the "proper" way to install drivers that have to compile from
        source on network boot images.
        
        Sent from sourceforge.net because you indicated interest in
        https://sourceforge.net/p/xcat/bugs/4457/
        
        To unsubscribe from further messages, please visit
        https://sourceforge.net/auth/subscriptions/
        
        Related
        
        Bugs: ~~#4457~~
        
        alternate
        
        graycol.gif
        
        If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

zhao er tao - 2014-12-10

After some investigation, I found this issue "/dev/fd/62: No such file or directory" is very difficult to be fixed in postinstall script of genimage. There are 2 options I can think to work around this issue:

use postscripts

install the cuda onto a diskfull node, and then merge all the files, directories, drivers, scripts to the rootimg created by genimage.

We will need to work with cuda team on how to deal with this issue. So I move this defect to 2.9.1.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Guang Cheng Li - 2015-03-09

Milestone: 2.9.1 --> 2.10
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Guang Cheng Li - 2015-03-09

We are working the GPU support in 2.10, moving this bug to 2.10

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

zhao er tao - 2015-04-15

status: open --> pending
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

zhao er tao - 2015-04-15

Hi, Ralph, xCAT2.10 have supported cuda installation, will you pls reference doc https://sourceforge.net/p/xcat/wiki/xCAT_P8LE_cuda_installing for more details.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

If I put the cuda install into the otherpkgs file and attempt an install I get the following error:

Adding system user `nvidia-persistenced' (UID 105) ...
Adding new group `nvidia-persistenced' (GID 110) ...
Adding new user `nvidia-persistenced' (UID 105) with group `nvidia-persistenced' ...
Not creating home directory `/'.
Loading new nvidia-346-346.46 DKMS files...
First Installation: checking all kernels...
It is likely that 3.16.0-30-generic belongs to a chroot's host
Building for architecture ppc64el
/usr/sbin/dkms: line 1903: /dev/fd/62: No such file or directory
/usr/sbin/dkms: line 1834: /dev/fd/62: No such file or directory
dpkg: error processing package nvidia-346 (--configure):
 subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of nvidia-346-uvm:
 nvidia-346-uvm depends on nvidia-346 (>= 346.46); however:
  Package nvidia-346 is not configured yet.

...
dpkg: error processing package libatk-wrapper-java-jni:ppc64el (--configure):
 dependency problems - leaving unconfigured
Processing triggers for libc-bin (2.19-10ubuntu2) ...
Processing triggers for initramfs-tools (0.103ubuntu8) ...
update-initramfs: Generating /boot/initrd.img-3.16.0-23-generic
df: Warning: cannot read table of mounted file systems: No such file or directory
mount: failed to read mtab: No such file or directory
Processing triggers for sgml-base (1.26+nmu4ubuntu1) ...
Errors were encountered while processing:

This is likely due to the /proc and /sys file systems not being mounted in the "chroot" directory during the otherpkgs script.

zhao er tao - 2015-04-20

Which version of xCAT are you using? The cuda installing function is supported in build later than Apr 9. Will you pls use the xcat snapshot from the link http://sourceforge.net/projects/xcat/files/ubuntu/devel/core-debs-snap.tar.bz2 for your verification?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

ralph bellofatto - 2015-04-18

This problem also exists for installs that have to be done in the postscrpt due to special the need to do "debconf-set-selections" to set preseed values for an install.

Any driver package that uses dkms will have this problem.

I suspect that the genimage script does the following somewhere for the processing of the pkglist.

We do have a workaround for this kind of problem, by putting following prefix into our postinstall script, we are able to get dkms enabled driver installs to work. However, we view this problem as a bug in the genimage code.

I suspect that the otherpkg and postinstall scripts should be run with these mounts present, or at least the same equivalent that genimage is using for the pkglist.

function finish { set +e; # pull the mounts out submounts to the install directory from the # /proc/mounts file. grep $installroot /proc/mounts | perl -nle '/(\S+) (\S+)/ && print "$2"' | tac | xargs umount #[ -n "$proc_dir" ] && umount $proc_dir; #[ -n "$sys_dir" ] && umount $sys_dir; } trap finish EXIT # # make sure proc and sys are available to anything running under chroot. # mount -o bind /sys $installroot/sys && sys_dir=$installroot/sys mount -o bind /proc $installroot/proc && proc_dir=$installroot/proc

Any post script that does an install that uses dkms (or any script that uses the same shell techniques) will fail.

The specific section of dkms that fails without the /proc and /sys file systems mounted is:

module_status_weak() { [[ $weak_modules ]] || return 1 local weak_ko mod installed_ko f ret=1 oifs=$IFS return $ret } do_status_weak() { local mvka m v k a kern status while read status mvka; do IFS=/ read m v k a kern <<< "$mvka" echo "$m, $v, $k, $a: installed-weak from $kern" done < <(module_status_weak "$@") } show_status() { do_status_weak "$module" "$module_version" "${kernelver[$j]}" "${arch[$j]}" } show_status

The specific file io redirection done by dkms and illusrated here FAILS to work if the /proc or /sys file system is not mounted.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

zhao er tao - 2015-04-20

For pkglist and otherpkglist which need to be done in genimage, the /proc of the host have been mounted to the chroot environment(modified in the build I mentioned in the above comment), but for postinstall, the admin need to mount manually since the code doesn't know whether it need to mount it.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

ralph bellofatto - 2015-04-20

Is there instructions for this requirement on line for doing post install scripts for genimage.

It would be picitularly useful if a search for xCAT and dkms would find the instructions for setting postinstall scripts such as these.

The previous advice that had been handed out (I'm not sure by who) was install and compile the cuda drivers during a postboot script.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

ralph bellofatto - 2015-04-20

I performed a yum update xCAT on 2.10 and retrieved the following version:

Version 2.10 (git commit cc73b77243db41fe256aa660740804d2c8144891, built Wed Apr 1 21:24:16 EDT 2015)

I'm still getting this when placing cuda in the "otherpkgs" list

First Installation: checking all kernels... It is likely that 3.16.0-30-generic belongs to a chroot's host Building for architecture ppc64el /usr/sbin/dkms: line 1903: /dev/fd/62: No such file or directory /usr/sbin/dkms: line 1834: /dev/fd/62: No such file or directory dpkg: error processing package nvidia-346 (--configure): subprocess installed post-installation script returned error exit status 1 dpkg: dependency problems prevent configuration of nvidia-346-uvm: nvidia-346-uvm depends on nvidia-346 (>= 346.46); however: Package nvidia-346 is not configured yet.

We are pointed to the following source forge repo:

baseurl=https://sourceforge.net/projects/xcat/files/yum/devel/core-snap

and

baseurl=https://sourceforge.net/projects/xcat/files/yum/xcat-dep/rh7/ppc64
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- zhao er tao - 2015-04-21
  
  Hi，Ralph, sorry for not doing Redhat xcat build that include the cuda supporting. The latest xcat-core build for Redhat is available at http://sourceforge.net/projects/xcat/files/yum/devel/core-rpms-snap.tar.bz2, will you pls have a try? Pls let me know if there is any issue.
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

ralph bellofatto - 2015-04-21

Redhat? This problem is on ubuntu. We won't get redhat with cuda until RHEL7.2 probably in October of this year.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- zhao er tao - 2015-04-23
  
  Yes, what I mean for Redhat xcat build is the OS of the Management Node, it can be RH6.x or RH7.x ppc64.
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

I did an update of xcat and re-ran the test,

This time, the build did not fail with the dkms error however, I did not get all of the nvidia drivers either.

on the genimage disk i get:

[root@dccxcat tulgpu-ub1410-nb-cuda01]# find rootimg/lib/modules/ -name 'nvidia*'
rootimg/lib/modules/3.16.0-23-generic/kernel/drivers/net/ethernet/nvidia
rootimg/lib/modules/3.16.0-23-generic/kernel/drivers/video/fbdev/nvidia
rootimg/lib/modules/3.16.0-23-generic/kernel/drivers/video/fbdev/nvidia/nvidiafb.ko
rootimg/lib/modules/3.16.0-30-generic/kernel/drivers/net/ethernet/nvidia
rootimg/lib/modules/3.16.0-30-generic/kernel/drivers/video/fbdev/nvidia
rootimg/lib/modules/3.16.0-30-generic/kernel/drivers/video/fbdev/nvidia/nvidiafb.ko

But on a diskful install I get:

root@tulgb004:/install# find /lib/modules -name nvidia*
/lib/modules/3.16.0-23-generic/updates/dkms/nvidia-346-uvm.ko
/lib/modules/3.16.0-23-generic/updates/dkms/nvidia_346.ko
/lib/modules/3.16.0-23-generic/kernel/drivers/net/ethernet/nvidia
/lib/modules/3.16.0-23-generic/kernel/drivers/video/fbdev/nvidia
/lib/modules/3.16.0-23-generic/kernel/drivers/video/fbdev/nvidia/nvidiafb.ko
/lib/modules/3.16.0-30-generic/updates/dkms/nvidia-346-uvm.ko
/lib/modules/3.16.0-30-generic/updates/dkms/nvidia_346.ko
/lib/modules/3.16.0-30-generic/kernel/drivers/net/ethernet/nvidia
/lib/modules/3.16.0-30-generic/kernel/drivers/video/fbdev/nvidia
/lib/modules/3.16.0-30-generic/kernel/drivers/video/fbdev/nvidia/nvidiafb.ko

So something is still off here.

zhao er tao - 2015-04-23

Can you show me the osimage object definition you used for genimage? The copycds can not generate cudafull or cudaruntime netboot osimage object for ubuntu 14.10. You need to generate it by following the cuda installation doc.

Last edit: zhao er tao 2015-04-23

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

xcat for ubuntu 14.10 won't run driver postinstall compile on genimage

An extreme cluster/cloud administration toolkit

Milestone

Searches

Help

#4457 xcat for ubuntu 14.10 won't run driver postinstall compile on genimage

Related

Discussion

make sure the debian front end does not produce this error:

debconf: (Can't locate Debconf/FrontEnd/Newt.pm in @INC

try this from: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=439763

Related

make sure the debian front end does not produce this error:

debconf: (Can't locate Debconf/FrontEnd/Newt.pm in @INC

try this from: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=439763

Related

make sure the debian front end does not produce this error:

debconf: (Can't locate Debconf/FrontEnd/Newt.pm in @INC

try this from: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=439763

Related