Menu

#58 "ipmiutil reset -D" doesn't soft shutdown specific hardware.

ipmiutil-3.1.x
closed
None
1
2023-03-13
2023-01-27
Naveen
No

Hi Andy Cress,

Recently after the upgrade from 2.7.9 to 3.1.5, device is not shutting down in specific linux machines during ipmiutil reset -D. I tried to go through the code to understand the differences which might be causing the issue. I found out that we use ipmi_cmd_mc() in 3.1.5 instead of ** ipmi_cmd()** in 2.7.9

Could you please provide your insights on this issue, any pointers where to look at?

Attaching command's Debug

2 Attachments

Discussion

  • Naveen

    Naveen - 2023-01-27

    From deeper inspection, the following function ** is_romley(vendid,prodid) is returning true, and giving the same hardware a different platform** for the same hardware in 3.1.5

     
  • Naveen

    Naveen - 2023-01-27

    and this change **(((bpower >= 5) && (platform == platIntel)) ** is restricting watchdog to run as part of ipmi_reset()

     
  • Andy Cress

    Andy Cress - 2023-01-27
    • Description has changed:

    Diff:

    --- old
    +++ new
    @@ -1,6 +1,6 @@
     Hi Andy Cress,
    
    -Recently after the upgrade from 2.7.9 to 3.1.5, device is not shutting down in specific linux machines during  ***ipmiutil sel -D***. I tried to go through the code to understand the differences which might be causing the issue. I found out that we use **ipmi_cmd_mc()** in 3.1.5 instead of ** ipmi_cmd()**  in 2.7.9
    +Recently after the upgrade from 2.7.9 to 3.1.5, device is not shutting down in specific linux machines during  ***ipmiutil reset -D***. I tried to go through the code to understand the differences which might be causing the issue. I found out that we use **ipmi_cmd_mc()** in 3.1.5 instead of ** ipmi_cmd()**  in 2.7.9
    
     Could you please provide your insights on this issue, any pointers where to look at?
    
     
  • Andy Cress

    Andy Cress - 2023-01-27

    It looks like there is a different in the driver type (openipmi vs. direct):
    In the 2.79 debug it opens the MV openipmi driver ok:
    ~~~ipmi_open: driver type = unknown
    ipmi_open_mv: cannot open /dev/ipmi/0
    ipmi_open_mv: succesfully opened /dev/ipmi0, fd=4
    ipmi_open rc = 0 type = open
    Driver type open, open rc = 0

    But in the 3.15 debug, it fails to open MV openipmi and falls back to directio.
    

    ipmi_open: driver type =
    ipmi_open_mv: cannot open /dev/ipmi/0
    ipmi_open_mv: cannot open /dev/ipmi0
    ipmi_open_mv: cannot open /dev/ipmidev0
    ipmi_open_mv: cannot open /dev/ipmidev/0
    imbapi ipmi_open_ia: open(/dev/imb) failed, No such file or directory
    smbios: Driver=7(KCS), sa=20, Base=0x0ca2, Spacing=1
    BMC KCS Initialized at 0x0ca2
    ipmidir Cmd=01 NetFn=06 Lun=00 Sa=20 Data(0):
    ~~~
    It may be that the directio commands start failing because the openipmi driver and directio are both contending for the same IO responses.
    Can you verify whether the openipmi driver is loaded for the 3.1.5 case?
    lsmod |grep ipmi

     
  • Andy Cress

    Andy Cress - 2023-01-27
    • assigned_to: Andy Cress
     
  • Naveen

    Naveen - 2023-01-27

    Hi Andy,
    After the upgrade , we moved from drivermode to driverless mode. so it was expected that open ipmi driver not being there. I tried to fallback to driver mode by creating /dev/ipmi0 . but i got the same output even after doing so.

    The issue seems to have originated from platform type. As platform was not platIntel, it missed to set watchdog and moved to chassis_ctl command and the corresponding output is seen in 3.1.5 log. Is there any reason you could think of, why the particular command seen in 3.1.5 log?

     
  • Andy Cress

    Andy Cress - 2023-01-27

    Changing from ipmi_cmd to ipmi_cmd_mc is what allows this to run either normally or via IPMB.

    For the command that repeats, It is a Chassis Control soft-shutdown command.
    ipmidir Cmd=02 NetFn=00 Lun=00 Sa=20 Data(1): 05
    Send Netfn=00 Cmd=02, raw: 00 20 00 02 05
    ipmidir Resp(1,2): status=0 cc=cf, Data(0):

    28.3 Chassis Control Command (0x02)
    0h = power down
    1h = power up
    2h = power cycle
    3h = hard reset
    4h = pulse Diagnostic interrupt
    5h = soft-shutdown of OS via ACPI

    The fact that this returns completion code 0xCF is the problem. The firmware says that it supports the ACPI soft shutdown.

    Two things to try:
    1) with ipmi driver
    To load the ipmi driver, you need more than the /dev node. It comes with Linux OS, so you can try this and see if that changes the symptoms:
    modprobe ipmi_devintf
    modprobe ipmi_si

    2) Revert to the older ipmiutil-2.7.9
    I suspect this will behave the same as the newer version.

     
  • Naveen

    Naveen - 2023-01-30

    Hi Andy,

    How significant is is_romley() check in ireset.c?
    I see that problem can be resolved by removing this check.

     
  • Andy Cress

    Andy Cress - 2023-01-30

    Good question. Originally the Romley systems had different soft shutdown support. The Intel Romley servers were from around 2010 IIRC, so it is possible Intel has reused the Romley ID for a different model, but they've never done that before. In any case, we need to remove the is_romley check in ireset.c based on your experience.
    If you do that on your system, it works, right?

     
  • Naveen

    Naveen - 2023-01-31

    Yes, it works without that check.

     
  • Andy Cress

    Andy Cress - 2023-02-17

    OK. I have been preparing a new ipmiutil-3.1.9 release, and I have added this change to ireset.c in it. Can you try it and verify that it resolves this problem?
    https://ipmiutil.sourceforge.net/FILES/ipmiutil-3.1.9.tar.gz
    I provided the source tarball, since I don't know which OS flavor you are running.

     
  • Andy Cress

    Andy Cress - 2023-02-17
    • status: open --> pending
     
  • Naveen

    Naveen - 2023-03-02

    Hi Andy, Sorry for the delayed response.

    When i compile and load this on my device, I am getting some issues regarding shared libraries, but I have tested removing the is_romley check in 3.1.5 and shutdown is working fine. I see that you have made it obselete in the code. so it must work.

     

    Last edit: Naveen 2023-03-02
  • Andy Cress

    Andy Cress - 2023-03-13

    The fix for this is now published in the ipmiutil-3.1.9 release at http://ipmiutil.sf.net.

     
  • Andy Cress

    Andy Cress - 2023-03-13
    • status: pending --> closed
     

Log in to post a comment.

MongoDB Logo MongoDB