#386 power supply displays "ok", when detailed info contains "failure detected" (power supply unplugged)
In a previous version (1.8.11) the power supply indication was fine, but in the 1.8.15 there an issue.
So when I query the sensor status: ipmitool sensor
and I get this:
Power Supply 1 | 0 | Watts | ok | na | na | na | na | na | na
Power Supply 2 | 130 | Watts | ok | na | na | na | na | na | na
, but when I query explicitly the power supply 1, I get this:
ipmitool sensor get "Power Supply 1"
Locating sensor record...
Sensor ID : Power Supply 1 (0x4)
Entity ID : 10.1 (Power Supply)
Sensor Type (Discrete): Power Supply (0x08)
Sensor Reading : 0 Watts
Event Message Control : Entire Sensor Only
States Asserted : Power Supply
[Presence detected]
** [Failure detected]**
OEM : 1
Hello,
please, can you add at which platform? Perhaps this is platform-specific issue and it will make debugging (for whoever is going to tackle this issue) esier.
Thanks!
OS: HELiOS release 6.5.3 (Carbon)
HW: ProLiant ML350 G6
If you need more, feel free to ask
The "ok" reported by "ipmitool sensor" indicates hat the sensor value was able to be successfully read and the power supply sensor returned a value of zero. "ipmitool sensor" does not report the decoded asserted sensor states, this is what "ipmitool sensor get" reports. To my knowledge, this is the way it has always worked in ipmitool.
On 1.8.11, it reports correctly "cr" for that situation, and "nc", when the power supply is plugged in.
But it is hard to tell, because "ipmitool sensor" has no column headers.
I had to go back and read the code in 1.8.11 and 1.8.15 to recall why things are different in 1.8.15.
The code in 1.8.11 was incorrectly decoding every sensor it saw as an Analog Sensor. The code did not correctly handle the fact that there are both Analog and Discrete Sensor with different decoding rules. If you have both 1.8.11 and 1.8.15, look at the differences for all the sensors using ipmitool -v sensor list and you will see quite a difference, 1.8.15 is doing the right thing. Power Supplies in your specific case are repoted as Discrete Sensors, not Analog Sensors.
In 1.8.11, here is how the Power Supply is being decoded (ipmitool -v sensor list), and this is incorrect as the Power Supply is a Discrete Sensor so it doesn't have any Thresholds.
Sensor ID : Power Supply 1 (0x3)
Entity ID : 10.1
Sensor Type (Analog) : Power Supply
Sensor Reading : 40 (+/- 0) Watts
Status : Lower Non-Critical
Lower Non-Recoverable : na
Lower Critical : na
Lower Non-Critical : na
Upper Non-Critical : na
Upper Critical : na
Upper Non-Recoverable : na
In 1.8.15, here is the correct decoding of the Discrete the Power Supply Sensor.
Sensor ID : Power Supply 1 (0x3)
Entity ID : 10.1
Sensor Type (Discrete): Power Supply
Sensor Reading : 45 Watts
States Asserted : Power Supply
[Presence detected]
Since Discrete Sensors can have multiple asserted states, there was no compatible a way to display a multi-state output with singlestate fields in ipmitool sensor list. The output for Discrete sensors simply reports the success of the sensor reading. You have to use ipmitool -v sensor list or ipmitool sensor get "sensor name" to get asserted states.
I hope you're not trying to suggest, that ok value in the 4th column should be interpreted as "value successfully read". I don't understand how this discrete-analog thing affects status reporting.
If a discrete sensor can have multiple asserted states, and any of them implies critical condition, the value should be critical.
I am saying value in the 4th column means that the the sensor was succesfully read and that you can rely upon the reported sensor value. In the case of Threshold sensors, you will get a terse mapping of the possible event states, or "ok" if none of of the event states are set.
Given the number of possible sensor types and the number of associated discrete sensor states, it is not at all clear to me that it is possible to map every reported state to "implies critical condition". Take a look the ipmi_event_sensor_types mappings in ipmi_sel.h and see how you might classify each of the possible IPMI_EVENT_CLASS_DISCRETE and IPMI_EVENT_CLASS_DIGITAL Event states to "critical condition".
I think it would be reasonable to report every sensor state as a set of comma separated strings in the 4th column, but this is going to cause problems with existing code that is currently attempting to interprete the 4th column programtically.
Sorry if this is not the right place to ask
My original intent was to display, whether
Is there a command to tell me, if power status is ok or not? (for a program to parse) The same applies to other categories. Until now I did the following: ran ipmitools status, and parsed the 4th field, grouped by units. Power is ok, if everything, which is in Watt is ok. Now I am unsure what the colums mean, becasue there is no column headers, so I need another way to dig out those infos.
Ok, how do you explain this:
Aggregated power supplies status displays 0x0280, but individual power supplies says ok.
Power Supply 1 | 0 | Watts | ok | na | na | na | na | na | na
Power Supply 2 | 110 | Watts | ok | na | na | na | na | na | na
Power Supplies | 0x0 | discrete | 0x0280| na | na | na | na | na | na
[root@pfltn175 ~]# ipmitool -V
ipmitool version 1.8.14
However, those 2 are not different (discrete, power supply type)
[root@pfltn175 ~]# ipmitool sensor get "Power Supplies" Locating sensor record... Sensor ID : Power Supplies (0x5) Entity ID : 10.3 (Power Supply) Sensor Type (Discrete): Power Supply (0x08) Sensor Reading : 0h Event Message Control : Entire Sensor Only States Asserted : Redundancy State [Redundancy Lost] OEM : 0 [root@pfltn175 ~]# ipmitool sensor get "Power Supply 1" Locating sensor record... Sensor ID : Power Supply 1 (0x3) Entity ID : 10.1 (Power Supply) Sensor Type (Discrete): Power Supply (0x08) Sensor Reading : 0 Watts Event Message Control : Entire Sensor Only States Asserted : Power Supply [Presence detected] [Failure detected] OEM : 1Sorry it took so long to get back on this, I was out all last week.
Power Supply 1 and Power Supply 2 both provide an analog reading. In the case above, Supply 1 is showing 0 Watts and Supply 2 is showing 110 Watts and both readings are valid so "ok" is displayed. In the case of the Power Supplies Sensor, there is no analog reading associated with the sensor, hence the 0x0 and "discrete" display. SInce there is no analog value to be read, it displays the non-decoded discrete bits (which sensor get decodes as Redundance State, [Redundance Lost]).
The only way that I know to get the the info you want is to look at the "States Asserted" in the individual sensor records or assign failing status to specific analog sensor readings. i.e. A Power Supply reporting 0 Watts is not providing any power.
What are your thoughts on doing the following:
I think it would be reasonable to report every sensor asserted state as a set of comma separated strings in the 4th column, but this is going to cause problems with existing code that is currently attempting to interprete the 4th column programtically.
Ok. I am now confused.
Is power supply an analog or a discrete sensor? You said:
and now you say:
That's what I expect in the fourth column.
I am here exacly because of compatibility problems. As I stated, in 1.8.11, the fourth column correctly (for me) displayed critical and not critical states, when the power supply was unplugged or plugged respectively. Our code just broke because of this change. That's why I would like to ask:
1.Could you provide me column headers / descriptions for the values in the output of
ipmitool sensoripmitool sdr type "Power Supply"|tail -1|grep -q "Fully Redundant".But as you mentioned, that a lot of things changed between .11 and .14, I am not sure, that our logic works as before for fans, temperature, voltage.
Here is a possible solution to enable you to easily script. The though would be to map all the possible discrete assertion states to either "ok" or "err" in order to provide a single value for column 4. It is not going to be perfect since there has to be a ok/err mapping for every possible assertion state.
In the case of a Threshold sensor, there would be no change, the colums are as follows for "sensor list"
nr - Non Recoverable
cr - Critical
nc - Non Critical
ns - not specified (means Value could not be read)
na - Not available
Name | AnalogValue | Units | [ok|nr|cr|nc|ns] | lnr value | lcr value | lncr value | uncr value | uc value | unr value
In the case of a Discrete sensor, the columns woud become as follows for "sensor list"
ok - No Error Assertion was identified by ipmitool
err - Error Assertion was identified by ipmitool
na - Not available
Name | [AnalogValue | 0xDiscreteValue] | [Units|"discrete"] | [ok|err] | na | na | na | na | na | na
What do you think?
I am generally fine with that, but what about backward compatibility you cared about? Would it be confusing to have the same states for discrete sensors (nr, cr, nc, ok) as the Thresohold sensors? Threshold sensors can also have ok status, you missed that state!
If we want to maintain compatibility failed sensor states could simply be marked as 'cr' instead of 'err'. I don' treally see a reasonable mapping to 'nr', 'nc'' or 'ns'.
Taking example the redundant power supply example: If I have 2 power source, they are fully redundant, one of them failed, the "Power supplies" should be critical, or not critical? (ok, in this example, the real true critical, is when all power supplies are failed)
I understand your example for redundant power supplies, but in my opinion, ipmitool should not be interpretating sensor information and doing so leads to these kind of issues. The verbose sensor information provides you the detail you need, you just need to look there if you want the correct interpretation. Reporting critical in the terse output would provide you information that there is some kind of failure and you could drill down as to what the exact failure by using verbose output. This would be better than what is currently done. I had proposed using the new value, "err" in the teres output to provide a very generic "somthing is wrong".
Hello,
please, what's the status of this ticket? Can it be closed?
Thanks,
Z.
I'm closing this as won't fix since there was no reply for almost 2 moths and conversation died out almost a year ago.