FGData Merge Request #252: Fix Equipment-Instrument Settings inconsistency toward Altimeter Indicator. (merged)

Altimeter Indicator seems to respect even rounding. Equipment-Instrument Settings do not. Thus, many times happens that altimeter indicator setting differs steadily by 0.01 from what Equipment-Instrument shows or what user sets if he uses only 2 digits. By this we create even rounding on both places so, both places shows the same value.
(Tested on release/2020.3.11 with Cessna 172P)

[James Turner]

I'd prefer to fix this by adding some helpers the Nasal/math.nas to do rounding to an explicit precision.

Eg Math.roundToDecimals(x, 2)

This would keep the code in the dialog.xml cleaner, and also make the intent clearer as well.

[Jimis Hol]

I can not develop a helper. The merge request can be used till some helper is involved. But, I would like to add that i am not 100% pleased by this fix that just follows the existing code. I assume that inHg in setting needs a third decimal, that can be added before a possible merge. 0.01 inHg equals to 3 hPa, so, as it is now, it can happen user not to see any change in inHg by adding 1 Hpa. With the addition of a third decimal he will see that changing.
If you agree with the above
setprop("/instrumentation/altimeter/setting-inhg-formatted", int(getprop("/instrumentation/altimeter/setting-inhg")1000.0+0.5)/1000.0+0.00001)
seems to do the job.
Still, because of the incremental feature on editing the formatted value, not allowing to set to a specific value -only giving the impression it does, till one tries to increase his just inserted new value where it becomes obvious that the old value is increased-, the third digit creates his own ambiguities to players.

[James Turner]

I've done some research and made some core changes: we already have:

math.round(a,b)

Where b is a scaling term, eg 'math.round(1239, 10.0)' will round to the nearest multiple of 10.

What I've done is extend math.floor() to support this optional second parameter, and add math.trunc() which also supports it.

math.floor is a true floor: always returns the integer value smaller than the argument. math.trunc always truncates towards zero, even for negative numbers.

I hope with these additions, the Nasal code can now become very simple and readable: no need to add +0.5 anywhere if we are lucky.

[Jimis Hol]

Considering https://wiki.flightgear.org/Nasal_library complete, i missed https://wiki.flightgear.org/Nasal_library/math A link from first to second in wiki might be useful to me.
I cannot tell where could be needed the new floor and trunc math functions, but in our case math.round function can be used. I updated the request accordingly, but i preferred to add a third digit in inHg for the reasons i wrote above.
Alter 0.001 to 0.01 if you do not like the 3nd digit.

[James Turner]

Actually the +0.00001 is still not ideal, in this case I /think/ we could use sprintf() which returns a string and hence avoids any further representation-as-double issues.

Eg this might look like:

setprop("/instrumentation/altimeter/setting-hpa-formatted", sprintf("%4d", getprop("/instrumentation/altimeter/setting-hpa"));

[Jimis Hol]

I refreshed the commits.
Actually what needed additional decimal was hPa because 0.01 inHg is about 1/3 fraction of 1Hpa. Eventually the same logic has the HI Offset where a user might see that he set for example 2 offset at his cockpit but be aware that he actually flights with 2.9 offset, when he reads the additional decimal in instruments popup window.