lcms-user

[Lcms-user] Configuration of the bounded/unbounded mode

[Sergey B. <byl...@am...>]

Hello, Colors Experts!

I am working on the OpenJDK project and have a couple of questions about the LittleCMS library.

The OpenJDK uses the LittleCMS for color conversion for a long time. And one of the use cases is 
getting the pixel data from the user in float format. Unfortunately before passing this float data 
to the LittleCMS the OpenJDK converts them to shorts(16 bit) and after color transformation converts 
the shorts to floats back and returns that floats to the user.

The current implementation was done for LittleCMS 1 and was not updated for version 2. I have read 
this[1] document about the "Floating Point Encoding Range" added to the spec in 2006 and how it is 
implemented in the LittleCMS library and would like to use this functionality in OpenJDK.

The change in OpenJDK should be straightforward, delete the intermediate conversion from floats to 
shorts and back and pass the floats directly to the LittleCMS library. But this is a moment I would 
like to clarify some possible impacts on the compatibility with the old behavior.

1. It is a good thing to do all intermediate computation in the unbounded mode during color 
transformation, but how the out-of-gamut values are handled for input and output data? That was not 
a problem for short but could be a  problem for floats, for example, what will happen if 2.0f is 
passed as a color value for the sRGB format - will it be clipped? or error will occur? or 
transformation will be applied as-is?

2. What about the output? There is a cmsFLAGS_NONEGATIVES flag to prevent negative values but when 
and how that values will be clipped by the upper bounds? There are no options to configure that and 
it seems some clipping occurs here and there in the code based on some constants which are not 
exposed as CMS API.

3. What about the use case when the user splits the "big" transform into a few small intermediate 
transformations, is it possible to enable "unbounded" mode for all transformations except for the 
output of the latest transform?

Thank you for any suggestions!

[1] https://www.littlecms.com/CIC18_UnboundedCMM.pdf

-- 
Best regards, Sergey.

Re: [Lcms-user] Configuration of the bounded/unbounded mode

[<mar...@li...>]

Hi,

Keep in mind the CMM internally operates always in unbounded mode and clips
values at final stage if you use anything but floating point. Also, not all
profiles allow unbounded mode. Doing the change you purpose would have a
severe impact on applications not expecting values negative or over 1.0. 
By using shorts, you are limiting precision to 16 bits, if you use floats,
you get more accuracy. Also note the encoding range changes, 0.1 for RGB,
0.100 for CMYK and for Lab it comes already decoded.
Unbounded mode works in both directions, if you feed the transform with a
value bigger then 1.0 and the profile supports it, the obtained XYZ would be
an out-of-gamut extrapolation.

The flag cmsFLAGS_NONEGATIVES was introduced to prevent only negative values
because security issues, does not work for values bigger than 1. In any
case, no matter what you do, the result would be wrong because this is an
out of gamut value. Clipping is an option, just the simplest gamut mapping
you could do.

> 3. What about the use case when the user splits the "big" transform into a
> few small intermediate transformations, is it possible to enable
"unbounded"
> mode for all transformations except for the output of the latest
transform?

Sure. Just ad a couple of "if >= 0 &&  <= 1.0" at the very end of the
pipeline.

Regards

Marti Maria
The LittleCMS Project
https://www.littlecms.com



> -----Original Message-----
> From: Sergey Bylokhov via Lcms-user <lcm...@li...>
> Sent: Sunday, August 13, 2023 1:48 AM
> To: lcm...@li...
> Subject: [Lcms-user] Configuration of the bounded/unbounded mode
> 
> Hello, Colors Experts!
> 
> I am working on the OpenJDK project and have a couple of questions about
> the LittleCMS library.
> 
> The OpenJDK uses the LittleCMS for color conversion for a long time. And
one
> of the use cases is getting the pixel data from the user in float format.
> Unfortunately before passing this float data to the LittleCMS the OpenJDK
> converts them to shorts(16 bit) and after color transformation converts
the
> shorts to floats back and returns that floats to the user.
> 
> The current implementation was done for LittleCMS 1 and was not updated
for
> version 2. I have read this[1] document about the "Floating Point Encoding
> Range" added to the spec in 2006 and how it is implemented in the
LittleCMS
> library and would like to use this functionality in OpenJDK.
> 
> The change in OpenJDK should be straightforward, delete the intermediate
> conversion from floats to shorts and back and pass the floats directly to
the
> LittleCMS library. But this is a moment I would like to clarify some
possible
> impacts on the compatibility with the old behavior.
> 
> 1. It is a good thing to do all intermediate computation in the unbounded
> mode during color transformation, but how the out-of-gamut values are
> handled for input and output data? That was not a problem for short but
> could be a  problem for floats, for example, what will happen if 2.0f is
passed
> as a color value for the sRGB format - will it be clipped? or error will
occur? or
> transformation will be applied as-is?
> 
> 2. What about the output? There is a cmsFLAGS_NONEGATIVES flag to prevent
> negative values but when and how that values will be clipped by the upper
> bounds? There are no options to configure that and it seems some clipping
> occurs here and there in the code based on some constants which are not
> exposed as CMS API.
> 
> 3. What about the use case when the user splits the "big" transform into a
> few small intermediate transformations, is it possible to enable
"unbounded"
> mode for all transformations except for the output of the latest
transform?
> 
> Thank you for any suggestions!
> 
> [1] https://www.littlecms.com/CIC18_UnboundedCMM.pdf
> 
> --
> Best regards, Sergey.
> 
> 
> _______________________________________________
> Lcms-user mailing list
> Lcm...@li...
> https://lists.sourceforge.net/lists/listinfo/lcms-user

Re: [Lcms-user] Configuration of the bounded/unbounded mode

[Philip R. <phi...@or...>]

Sergey,

Why do you want to do this ? I'm not sure I see the value in it.

Although JDK does support float for images, none of the standard image 
formats use floats.
So I'm not sure how well this change would be tested nor where we'd see 
any benefit from it
so who is asking for this ?

-phil.

On 8/13/23 2:36 AM, mar...@li... wrote:
> Hi,
>
> Keep in mind the CMM internally operates always in unbounded mode and clips
> values at final stage if you use anything but floating point. Also, not all
> profiles allow unbounded mode. Doing the change you purpose would have a
> severe impact on applications not expecting values negative or over 1.0.
> By using shorts, you are limiting precision to 16 bits, if you use floats,
> you get more accuracy. Also note the encoding range changes, 0.1 for RGB,
> 0.100 for CMYK and for Lab it comes already decoded.
> Unbounded mode works in both directions, if you feed the transform with a
> value bigger then 1.0 and the profile supports it, the obtained XYZ would be
> an out-of-gamut extrapolation.
>
> The flag cmsFLAGS_NONEGATIVES was introduced to prevent only negative values
> because security issues, does not work for values bigger than 1. In any
> case, no matter what you do, the result would be wrong because this is an
> out of gamut value. Clipping is an option, just the simplest gamut mapping
> you could do.
>
>> 3. What about the use case when the user splits the "big" transform into a
>> few small intermediate transformations, is it possible to enable
> "unbounded"
>> mode for all transformations except for the output of the latest
> transform?
>
> Sure. Just ad a couple of "if >= 0 &&  <= 1.0" at the very end of the
> pipeline.
>
> Regards
>
> Marti Maria
> The LittleCMS Project
> https://www.littlecms.com
>
>
>
>> -----Original Message-----
>> From: Sergey Bylokhov via Lcms-user <lcm...@li...>
>> Sent: Sunday, August 13, 2023 1:48 AM
>> To: lcm...@li...
>> Subject: [Lcms-user] Configuration of the bounded/unbounded mode
>>
>> Hello, Colors Experts!
>>
>> I am working on the OpenJDK project and have a couple of questions about
>> the LittleCMS library.
>>
>> The OpenJDK uses the LittleCMS for color conversion for a long time. And
> one
>> of the use cases is getting the pixel data from the user in float format.
>> Unfortunately before passing this float data to the LittleCMS the OpenJDK
>> converts them to shorts(16 bit) and after color transformation converts
> the
>> shorts to floats back and returns that floats to the user.
>>
>> The current implementation was done for LittleCMS 1 and was not updated
> for
>> version 2. I have read this[1] document about the "Floating Point Encoding
>> Range" added to the spec in 2006 and how it is implemented in the
> LittleCMS
>> library and would like to use this functionality in OpenJDK.
>>
>> The change in OpenJDK should be straightforward, delete the intermediate
>> conversion from floats to shorts and back and pass the floats directly to
> the
>> LittleCMS library. But this is a moment I would like to clarify some
> possible
>> impacts on the compatibility with the old behavior.
>>
>> 1. It is a good thing to do all intermediate computation in the unbounded
>> mode during color transformation, but how the out-of-gamut values are
>> handled for input and output data? That was not a problem for short but
>> could be a  problem for floats, for example, what will happen if 2.0f is
> passed
>> as a color value for the sRGB format - will it be clipped? or error will
> occur? or
>> transformation will be applied as-is?
>>
>> 2. What about the output? There is a cmsFLAGS_NONEGATIVES flag to prevent
>> negative values but when and how that values will be clipped by the upper
>> bounds? There are no options to configure that and it seems some clipping
>> occurs here and there in the code based on some constants which are not
>> exposed as CMS API.
>>
>> 3. What about the use case when the user splits the "big" transform into a
>> few small intermediate transformations, is it possible to enable
> "unbounded"
>> mode for all transformations except for the output of the latest
> transform?
>> Thank you for any suggestions!
>>
>> [1] https://www.littlecms.com/CIC18_UnboundedCMM.pdf
>>
>> --
>> Best regards, Sergey.
>>
>>
>> _______________________________________________
>> Lcms-user mailing list
>> Lcm...@li...
>> https://lists.sourceforge.net/lists/listinfo/lcms-user
>
>
> _______________________________________________
> Lcms-user mailing list
> Lcm...@li...
> https://lists.sourceforge.net/lists/listinfo/lcms-user

Re: [Lcms-user] Configuration of the bounded/unbounded mode

[Sergey B. <byl...@am...>]

On 8/13/23 15:07, Philip Race wrote:
> Sergey,
> 
> Why do you want to do this ? I'm not sure I see the value in it.
> 
> Although JDK does support float for images, none of the standard image
> formats use floats.

Unfortunately not, even though most standard images use bytes and ints to convert color profiles and 
access pixels in images, we use floats. So the common pattern we have is:
The actual data of the image(byte, short, int, float, double)->float->short->start lcms 
internal->float->apply color transform in floats->end lcms internals->short->float->the data of the 
image(byte, short, int, float, double). So at the end we have a "good" precision of the result 
especially if the image stores the color component in 17+ bits.

An examples of such conversion:
1. 
https://github.com/openjdk/jdk/blob/595fdd36c5f735b53ed2950c539be46382f9bcdd/src/java.desktop/share/classes/sun/java2d/cmm/lcms/LCMSTransform.java#L230
2. 
https://github.com/openjdk/jdk/blob/595fdd36c5f735b53ed2950c539be46382f9bcdd/src/java.desktop/share/classes/java/awt/image/ColorConvertOp.java#L809
3. Well we have a public API for such float based conversion - to/From/RGB/CIEXYZ: 
https://github.com/openjdk/jdk/blob/595fdd36c5f735b53ed2950c539be46382f9bcdd/src/java.desktop/share/classes/java/awt/color/ICC_ColorSpace.java#L202


But there are a few things I would like to check before starting to work on the OpenJDK side.
1. The slow performance of the float based calculations, described here: 
https://github.com/mm2/Little-CMS/issues/356 But I think elimination of an additional conversion 
should compensate that.
2. Compatibility related to the out-gamut values discussed in this thread. I would like to 
understand how to implement clip properly. Note that our current implementation do not clip 
out-of-gamtut values but blindly cast to short and get some "random" value at the end.

-- 
Best regards, Sergey.

Re: [Lcms-user] Configuration of the bounded/unbounded mode

[<mar...@li...>]

Hello Sergey,

> The actual data of the image(byte, short, int, float, double)->float->short->start
> lcms
> internal->float->apply color transform in floats->end lcms
> internal->float->internals->short->float->the data of the
> image(byte, short, int, float, double). So at the end we have a "good" precision
> of the result especially if the image stores the color component in 17+ bits.

I don't know the internal architecture of OpenJDK, so maybe this is not feasible, but it seems to me the way you are doing the conversions is a serious throughput killer.  Lcms goes in a lot of trouble to optimize color transforms to make them as fast as possible whilst keeping the maximum precision. So, if you are going to convert a bitmap of 8 bits to another bitmap of 8 bits, your best option is to tell lcms to use 8 bits on input and output. The math precision and throughput performance you will obtain is actually better than going to float and then back to 8 bits, because lcms use tricks on rounding and fixed-point interpolation to go fast and accurate. Ditto for 16 bits, float, double, half...

Floating point transforms should be only used when your data is already in floating point, or in the case you want to convert to floating point. But that should be transparent to you, you should only tell the engine how your data is organized and let it to pick the better way to do the conversion. If you always convert to float assuming this is going to increase precision you are obtaining the inverse effect: sometimes less precision and definitively slower throughput. But again, I don't know OpenJDK architecture, so maybe the float step is unavoidable because other components.

Best regards

Marti Maria
The LittleCMS Project
https://www.littlecms.com




> -----Original Message-----
> From: Sergey Bylokhov <byl...@am...>
> Sent: Monday, August 14, 2023 10:18 PM
> To: Philip Race <phi...@or...>; mar...@li...; lcms-
> us...@li...
> Subject: Re: [Lcms-user] Configuration of the bounded/unbounded mode
> 
> On 8/13/23 15:07, Philip Race wrote:
> > Sergey,
> >
> > Why do you want to do this ? I'm not sure I see the value in it.
> >
> > Although JDK does support float for images, none of the standard image
> > formats use floats.
> 
> Unfortunately not, even though most standard images use bytes and ints to
> convert color profiles and access pixels in images, we use floats. So the
> common pattern we have is:
> The actual data of the image(byte, short, int, float, double)->float->short->start
> lcms
> internal->float->apply color transform in floats->end lcms
> internal->float->internals->short->float->the data of the
> image(byte, short, int, float, double). So at the end we have a "good" precision
> of the result especially if the image stores the color component in 17+ bits.
> 
> An examples of such conversion:
> 1.
> https://github.com/openjdk/jdk/blob/595fdd36c5f735b53ed2950c539be46382f
> 9bcdd/src/java.desktop/share/classes/sun/java2d/cmm/lcms/LCMSTransform.j
> ava#L230
> 2.
> https://github.com/openjdk/jdk/blob/595fdd36c5f735b53ed2950c539be46382f
> 9bcdd/src/java.desktop/share/classes/java/awt/image/ColorConvertOp.java#L8
> 09
> 3. Well we have a public API for such float based conversion -
> to/From/RGB/CIEXYZ:
> https://github.com/openjdk/jdk/blob/595fdd36c5f735b53ed2950c539be46382f
> 9bcdd/src/java.desktop/share/classes/java/awt/color/ICC_ColorSpace.java#L20
> 2
> 
> 
> But there are a few things I would like to check before starting to work on the
> OpenJDK side.
> 1. The slow performance of the float based calculations, described here:
> https://github.com/mm2/Little-CMS/issues/356 But I think elimination of an
> additional conversion should compensate that.
> 2. Compatibility related to the out-gamut values discussed in this thread. I
> would like to understand how to implement clip properly. Note that our
> current implementation do not clip out-of-gamtut values but blindly cast to
> short and get some "random" value at the end.
> 
> --
> Best regards, Sergey.