slurm-roll / Discussion / General Discussion: using both slurm roll and SDSC Mellanox OFED roll

Noam Bernstein - 2019-06-20

Hi - is there any way to get the slurm roll to work with the SDSC Mellanox OFED roll? The slurm roll appears to require libibmad.so.5, while the OFED roll provides libibmad.so.12.

thanks, Noam
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Werner Saar - 2019-06-21
  
  Hi,
  
  the slurm roll don't provide libibmad.
  
  Best regards
  
  Werner
  
  On 6/20/19 9:30 PM, Noam Bernstein wrote:
  
  Hi - is there any way to get the slurm roll to work with the SDSC Mellanox OFED roll? The slurm roll appears to require libibmad.so.5, while the OFED roll provides libibmad.so.12.
  
  thanks, Noam
  
  using both slurm roll and SDSC Mellanox OFED roll
  
  Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/slurm-roll/discussion/general/
  
  To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
  - Noam Bernstein - 2019-06-21
    
    On Jun 20, 2019, at 10:50 PM, Werner Saar wernsaar@users.sourceforge.net wrote:
    
    Hi,
    
    the slurm roll don't provide libibmad.
    
    I know it doesn't provide it. I mean that it requires it:
    [bernadm @tin 090_slurm]$ rpm -q -R -p /export/rocks/install/rolls/slurm/7.0.0.191/x86_64/RedHat/RPMS/slurm-18.08.6-2.el7.centos.x86_64.rpm | grep libibmad
    libibmad.so.5()(64bit)
    libibmad.so.5(IBMAD_1.3)(64bit)
    
    That is indeed the version that's included with CentOS (as of the 7.6 update)
    [bernadm @tin 090_slurm]$ rpm -q -l -p /export/rocks/install/rolls/Centos_7_6/7.0/x86_64/RedHat/RPMS/infiniband-diags-2.0.0-2.el7.x86_64.rpm | grep libibmad
    /usr/lib64/libibmad.so.5
    /usr/lib64/libibmad.so.5.5.0
    
    But when I install the Mellanox OFED software, it installs libibmad.so.12,
    [bernadm @tin 090_slurm]$ rpm -q -l -p /export/rocks/install/rolls/mlnx-ofed-4.3-1.0.1.0-3.10.0-957.21.2.el7/7.0/x86_64/RedHat/RPMS/libibmad-5.4.0.MLNX20190423.1d917ae-0.1.46101.x86_64.rpm
    /usr/lib64/libibmad.so.12
    /usr/lib64/libibmad.so.12.0.0
    which then fails to satisfy the dependency requirement of the slurm rpm.
    
    Is the slurm requirement for that particular library version set when you compile it, or is there some way for me to change it when I install?
    
    thanks, Noam
    
    alternate
    
    If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
    - Werner Saar - 2019-06-22
      
      Hi,
      
      please read https://slurm.schedmd.com/quickstart_admin.html.
      
      Best regards
      
      Werner
      
      On 6/21/19 3:49 PM, Noam Bernstein wrote:
      
      On Jun 20, 2019, at 10:50 PM, Werner Saar wernsaar@users.sourceforge.net wrote:
      
      Hi,
      
      the slurm roll don't provide libibmad.
      
      I know it doesn't provide it. I mean that it requires it:
      [bernadm @tin 090_slurm]$ rpm -q -R -p /export/rocks/install/rolls/slurm/7.0.0.191/x86_64/RedHat/RPMS/slurm-18.08.6-2.el7.centos.x86_64.rpm | grep libibmad
      libibmad.so.5()(64bit)
      libibmad.so.5(IBMAD_1.3)(64bit)
      
      That is indeed the version that's included with CentOS (as of the 7.6 update)
      [bernadm @tin 090_slurm]$ rpm -q -l -p /export/rocks/install/rolls/Centos_7_6/7.0/x86_64/RedHat/RPMS/infiniband-diags-2.0.0-2.el7.x86_64.rpm | grep libibmad
      /usr/lib64/libibmad.so.5
      /usr/lib64/libibmad.so.5.5.0
      
      But when I install the Mellanox OFED software, it installs libibmad.so.12,
      [bernadm @tin 090_slurm]$ rpm -q -l -p /export/rocks/install/rolls/mlnx-ofed-4.3-1.0.1.0-3.10.0-957.21.2.el7/7.0/x86_64/RedHat/RPMS/libibmad-5.4.0.MLNX20190423.1d917ae-0.1.46101.x86_64.rpm
      /usr/lib64/libibmad.so.12
      /usr/lib64/libibmad.so.12.0.0
      which then fails to satisfy the dependency requirement of the slurm rpm.
      
      Is the slurm requirement for that particular library version set when you compile it, or is there some way for me to change it when I install?
      
      thanks, Noam
      
      using both slurm roll and SDSC Mellanox OFED roll
      
      Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/slurm-roll/discussion/general/
      
      To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/
      
      If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
      - Noam Bernstein - 2019-06-22
        
        On Jun 22, 2019, at 12:33 AM, Werner Saar wernsaar@users.sourceforge.net wrote:
        
        Hi,
        
        please read https://slurm.schedmd.com/quickstart_admin.html. https://slurm.schedmd.com/quickstart_admin.html.
        
        I have read that. In fact, my current workaround is to install your very helpful slurm roll for rocks integration, but then to build my own slurm RPMs from the slurm source (using the rpmbuild instructions from the quickstart admin link). Those then depend on the newer libibmad that I have installed. I then override the slurm roll RPMS with my own RPMs by putting them in /export/rocks/instal/contrib....RPMS.
        
        That works OK, but I was just hoping for a way to re-build your roll with my own RPMs. Are there any instructions for how to do that from sourceforce SVN source tree? I looked, but didn't see anything.
        
        thanks, Noam
        
        alternate
        
        If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
        
        Werner Saar - 2019-06-23
        
        Hi,
        
        attached is a step-by-step guide to rebuild slurm, the roll ISO
        
        and to reinstall slurm on the frontend.
        
        Best regards
        
        Werner
        
        On 6/22/19 7:14 PM, Noam Bernstein wrote:
        
        On Jun 22, 2019, at 12:33 AM, Werner Saar wernsaar@users.sourceforge.net wrote:
        
        Hi,
        
        please read https://slurm.schedmd.com/quickstart_admin.html. https://slurm.schedmd.com/quickstart_admin.html.
        
        I have read that. In fact, my current workaround is to install your very helpful slurm roll for rocks integration, but then to build my own slurm RPMs from the slurm source (using the rpmbuild instructions from the quickstart admin link). Those then depend on the newer libibmad that I have installed. I then override the slurm roll RPMS with my own RPMs by putting them in /export/rocks/instal/contrib....RPMS.
        
        That works OK, but I was just hoping for a way to re-build your roll with my own RPMs. Are there any instructions for how to do that from sourceforce SVN source tree? I looked, but didn't see anything.
        
        thanks, Noam
        
        using both slurm roll and SDSC Mellanox OFED roll
        
        Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/slurm-roll/discussion/general/
        
        To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/
        
        rebuild-slurm.txt
        
        If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

using both slurm roll and SDSC Mellanox OFED roll

Slurm Resource Manager for Rocks Clusters

Forums

Help

using both slurm roll and SDSC Mellanox OFED roll

using both slurm roll and SDSC Mellanox OFED roll

Slurm Resource Manager for Rocks Clusters

Forums

Help

using both slurm roll and SDSC Mellanox OFED roll document.SUBSCRIPTION_OPTIONS = { "thing": "topic", "subscribed": false, "url": "subscribe", "icon": { "css": "fa fa-envelope-o" } };

using both slurm roll and SDSC Mellanox OFED roll