#259 FreeImage_ConvertTo[float format] clamps HDR data to [0,1]

[Karl Schmidt]

FreeImage_ConvertToRGBF and friends will clamp any input outside of the [0,1] range. This has been the case since 3.16 and was caused by the changes in this revision.

Specifically in our case, FreeImage_ConvertToRGBF will clamp an input image in FIT_RGBAF format when it really only needs to drop the alpha channel. Which caused any of our test system renderings that used EXR light probes to suddenly become very dark.

Obviously this change in behavior was intentional on your part so it may be that we'll need to do this sort of conversion manually. It seems like an odd behavior, though: I would normally expect that a conversion from one HDR format to another would preserve the dynamic range unless there's a specific reason not to.

[Karl Schmidt]

Reading the documentation (always a plus) I see that the specified behavior for RGBAF to RGBF conversion is "For 128-bit RGBAF images, conversion is done by copying the source float pixel values into the destination float pixel values, skipping the alpha channel."

So perhaps not quite so intentional as I assumed. It would certainly be preferable from my perspective if the HDR-to-HDR conversions didn't rescale or clamp the data.

[Hervé Drolon]

Hi,

In order to be consistent, all float images in FreeImage should have a range in [0..1].
That means that some corrections should be done in current version
- FreeImage_ConvertToRGBF(sourceType == FIT_RGBF) should not call FreeImageClone but instead clamp input pixel values to [0..1]
- FreeImage_ConvertToRGBAF(sourceType == FIT_RGBAF) should not call FreeImageClone but instead clamp input pixel values to [0..1]
- FreeImage_ConvertToFloat(sourceType == FIT_FLOAT) should not call FreeImageClone but instead clamp input pixel values to [0..1]
- other behaviors should not be changed

I will make these corrections for the next release.

Hervé

[Karl Schmidt]

That's consistent, albeit surprising. :)

Does this mean that we cannot in general expect FreeImage to be well-behaved with any float image that has data outside of the [0,1] range? Very few EXR and HDR files in the wild will adhere to that limitation, nor will most applications that need to generate them (renderers that use them as pre-tonemapped intermediates in compositing, etc). The HDR formats are generally used because you actually need the range and chopping most of it off is a definite dealbreaker.

[Hervé Drolon]

Hi Karl,

I'm not a professional user of HDR formats (but use them extensively for image processing) ...
If you have a better solution that is consistent with all float conversion functions (and tone mapping functions), then please give us some advices and a better solution.

To my mind, a float format cannot be outsided a [0..1] range when applying an image processing function, but maybe I'm wrong ?

Maybe we could had a new "FreeImage_Normalize" function (to be used with FIT_FLOAT, FIT_RGBF, FIT_RGBAF types, maybe other types) to normalize images to [0..1] when needed ?

Hervé

[Karl Schmidt]

First, sorry, this ended up a bit more long-winded than I intended. :)

It varies but I would say that most image processing operations -- blends, filter kernels, histograms, etc -- are valid for the entire gamut of a colorspace and not just the [0..1] range. Some, like "1.0 - x" inversion, might not be all that meaningful in HDR and I'm sure some don't apply at all. I realize that you may not wish to always support that sort of behavior in FreeImage, but in this case FreeImage_ConvertToRGBF worked "correctly" for us before and this change was a regression, hence the report.

From my rendering-biased perspective I would say that typically an image is in an HDR format when it's intended for something other than direct display. The image values aren't intended to map directly to pixel intensities but instead represent something like the physical W/m² radiance observed by a camera. Still an image, often something you want to manipulate, but not something you can just send to your local GPU.

As a result I don't think you can make any general assumptions on the range for floating point image data. There are cases where NaN or Inf are perfectly valid (the observed radiance of the sun as stored in halfs, say). There are cases where negative numbers are perfectly valid. Inevitably there are corner cases where one has to assume something, but my expectation would be for operators on the image to preserve as much of the range as they can.

Practical examples:

The EXR format was at least originally created for compositing workflows in VFX. In that context it can for instance be commont to do separate render passes to store the light contributed from each light source in separate EXRs. This allows users to later change the color or intensity of a light by just manipulating that image, rather than doing (potentially very expensive) expensive rerendering. This sort of compositing requires the actual radiances be stored in the EXR. After tweaking the individual contributions you combine them for a final image, and only apply exposure and other tonemapping operations to compress the range to e.g. sRGB for display as the last step in the process. This is a workflow example in nvidia's iray.

The situation that prompted this report would be something like Paul Debevec's light probes which capture the lighting environment at a point in space and can then be used directly as light sources in a renderer. Specifically in our case, we had a probe in EXR format that FreeImage loaded as RGBAF, and we wanted to drop the alpha channel before passing it to our renderer since it wasn't meaningful in that context. Having that result in the radiance of the sun changing from 10^8 W/m² to 1 W/m² after upgrading to FreeImage 3.16 was unexpected, to say the least :)

There are other examples from photography and so on, but I know a lot less about that.

Suggestions:

There's no 100% fireproof way to deal with conversions from HDR to LDR data and vice versa. If you assume the range is [0..1] the conversion is trivial but it will be irrevesible and destructive for a very large number of typical HDR images and will break otherwise perfectly valid float-to-float operations if applied everywhere. An alternative is to let the user provide a tonemapping operator (and its reverse) that specifies how to compress and decompress the full HDR range they are interested in to LDR, but that's a big hassle, can be extremely lossy and might not always be possible.

I think the best that can be done is to preserve as much of the high dynamic range of the HDR format when possible and make some arbitrary-yet-sane assumption when you can't. So, something like
Float-to-float: Round to nearest target value. Undefined behavior if the source value is outside the range of the target format for e.g. double to half.
Float-to-fixed & fixed-to-float: Assume that the fixed point range represents [0..1], then as above.

[Roeland Schoukens]

Yes this is definitely a mistake. It's like arbitrarily clamping 8-bit images to [0, 42]. You just don't do it. One of the most important reasons people use HDR formats like EXR or HDR is exactly because they allow values outside [0, 1].

Another hint are the tonemapping functions (FreeImage_ToneMapping and friends). The whole point of those operators is to map values outside [0, 1] to LDR images.