YCCK to RGB and CMYK to RGB Conversion
SIMD-accelerated libjpeg-compatible JPEG codec library
Brought to you by:
dcommander
Menu
▾
▴
#15 YCCK to RGB and CMYK to RGB Conversion
This patch enhances libjpeg-turbo by adding YCCK to RGB and CMYK to RGB color conversion. There are some files that fail to decode to RGB with libjpeg-turbo and after applying these patches these files decode properly.
./djpeg -rgb < image_ycck.jpeg > image_ycck.ppm
./djpeg -rgb < image_cmyk.jpeg > image_cmyk.ppm
YCCK->RGB and CMYK->RGB
JPEG image with YCCK raw format
I've checked in a modified patch that fixes several issues:
(1) It now takes into account the libjpeg-turbo colorspace extensions.
(2) It uses MAXJSAMPLE instead of 255 (yeah, we don't really support 12-bit samples, but you never know whether upstream libjpeg may want to borrow this patch.)
(3) It doesn't use the output buffer as temp space. The submitted patch assumed a 32-bit output pixel, which isn't always true. If the output buffer were 24-bit, the submitted code would have overrun the buffer on the last pixel.
I observe that the routines still seem to produce RGB images that are too saturated.
Also, can you pls upload a CMYK test image. I have only been able to test YCCK, because that's what Photoshop generates.
I tried to search for the "right" formulas of doing these color conversions and failed to find a really relevant authoritative source of this information. Poynton Color FAQ suggests that these conversions may be not so easy [1]. Moreover, when I tried to provide a "review" [2] for the linaro libjpeg-turbo improvements which also included this patch, I noticed that the other existing patches to libjpeg floating around seem to be doing such conversion in a bit different way [3]. In my opinion, the right solution would be to invite all the interested parties (openoffice, maybe browsers, ...) in this YCCK/CMYK support discussion and solve the problem once and for all.
Also it's a bit strange for the patch originating from linaro to use double precision floating point calculations where they seem to be perfectly avoidable. Double precision floating point math is really slow on a lot of ARM processors, including ARM Cortex-A8.
And finally, it would be really nice to add support for these newly added formats to the test suite ('make test'). So that the potential bugs introduced by the possible future SIMD optimizations of this code could be easily identified.
I completely agree with you on all points. It's a bit of a tricky issue, because when I open a CMYK or YCCK JPEG image in another program to verify the accuracy of its color reproduction, I have to bear in mind that the other program is also doing some sort of CMYK-to-RGB conversion. I trust PhotoShop, so what I'm doing to check libjpeg-turbo's conversion algorithms is starting with a photo that I took and whose appearance is known to me.
I open the photo in PhotoShop, switch the mode to CMYK, and save as a JPEG image (this actually generates a YCCK JPEG image.) I save both a copy with the colorspace (SWOP) embedded in the JPEG file and another without the colorspace embedded. Here's what I observe when opening these in various programs:
YCCK JPEG with colorspace embedded:
PhotoShop: Correct
Preview: Appears washed out (equivalent of about 15% more brightness and 15% less contrast)
Graphic Converter: Appears washed out (equivalent of about 15% more brightness and 15% less contrast)
Firefox: Heavily over-saturated (particularly in the green tones)
IrfanView: Heavily over-saturated (particularly in the green tones)
Output of djpeg in libjpeg-turbo: Heavily over-saturated (particularly in the green tones)
YCCK JPEG without colorspace embedded:
PhotoShop: Correct
Preview: Appears slightly darker and less saturated
Graphic Converter: Appears slightly darker and less saturated
Firefox: Heavily over-saturated (particularly in the green tones)
IrfanView: Heavily over-saturated (particularly in the green tones)
Output of djpeg in libjpeg-turbo: Heavily over-saturated (particularly in the green tones)
Mozilla and Irfan are apparently doing the same thing we are (not surprising since both use libjpeg-turbo.) Apple's implementation is more correct (but still doesn't produce the output I would expect.) I understand that any talk of CMYK-to-RGB conversion is kind of meaningless without colorspaces, but probably we should be making assumptions that the YCCK/CMYK image uses a common colorspace like SWOP and that the output is sRGB.
Regarding adding this to 'make test', first we need to agree upon whether to remove double precision float arithmetic and replace it with integer arithmetic and on what formulas to use, but definitely it will be added to make test after that.
I think the best thing to do regarding precision is to create lookup tables like the code is already doing for YCbCr, so we can avoid the intermediate conversion steps.
http://software.intel.com/sites/products/documentation/hpc/ipp/ippi/ippi_ch15/functn_YCCKToCMYK_JPEG.html#functn_YCCKToCMYK_JPEG
In looking at the YCC->RGB conversion table, are the scalar offsets (as in the link above) taken into account? It doesn't quite look like it.
The math they're using to convert from YCCK to CMYK is the same as ours. The scalars in their formula are to convert from a range of 0 to 255 to a range of -128 to 127. libjpeg does that conversion whenever it builds the lookup tables, so it doesn't have to add the scalars later.
Our conversion routines seem to produce the same results (over-saturated output) whenever a CMYK JPEG (as opposed to YCCK) is used, which tells me that the problem is in the final CMYK-to-RGB conversion, not the intermediate YCCK-to-CMYK conversion. However, taking into account the fact that JPEG is using inverted CMYK, then our formulas are the same as the ones used by Mozilla (see https://bugzilla.mozilla.org/show_bug.cgi?id=44781).
R = ( 1 - ( C * ( 1 - K ) + K )) * 255
G = ( 1 - ( M * ( 1 - K ) + K )) * 255
B = ( 1 - ( Y * ( 1 - K ) + K )) * 255
are the formulas they report. If you change this to inverted CMYK with a range of 0-255, it reduces to the familiar R = C' K' / 255, G = M' K' / 255, B = Y' K' / 255.
I went ahead and changed our routines to use integer math. After quite a bit of poking around on the web, the best information I can get is that it's impossible to do any kind of "real" conversion without using a comprehensive color management system. I know PhotoShop does this, and I assume that Preview and Graphic Converter are hooking into the Apple CMS somehow. Obviously that's not something we could do in a low-level library. I note that Mozilla doesn't do CMYK-to-RGB conversion within libjpeg-- possibly for that reason.
I begin to wonder whether this feature has any use at all. Seems like, if I really wanted to decode a CMYK JPEG, what I'd do is decode YCCK to CMYK (which libjpeg can already do), then pass the CMYK through a color management library to get RGB.
Very keen to get opinions on this. I note that prominent open source projects, including ImageMagick and Mozilla, appear to be taking CMYK out of libjpeg and then converting it to RGB using another library that supports color management. That further bolsters the view that having these routines in libjpeg isn't useful.
I really want there to be a consumer, or even a potential consumer, for this for it to remain in the library. I'm OK with it being an uncorrected conversion, as long as someone can make a solid case for why such a conversion is useful to have in libjpeg.
I will chime in since I most recently commented here.
The reason I was going down this route is that netpbm libraries do not support 4 channel jpeg operations - and that is what I was using for resizing. I was attempting a shortcut around the deficiency in netpbm and found this patch ready on libjpeg-turbo.
In reality, I want to maintain the colorspace as much as I can, so I will end up using graphicsmagick to do resizes if I detect more than 3 channels until I find a suitable middle replacement between jpeg decode and encode.
So in the end, my case is not a deficiency of libjpeg, but of netpbm, so I'll say that it does not make sense in libjpeg.
Hi DRC,
I think it makes sense to have these conversions in libjpeg for the same reason because of which we have YUV to RGB conversion in libjpeg. I can tell you that there is major consumer tablet manufacturer who is using libjpeg library in their production devices with such conversions built in them. For confidentiality reason I can not disclose the name.
Here is one example of application:
Photo browsing application that need to show all kind of jpeg images. Given that there exist no standardize color conversion libraries, such applications entirely rely on libjpeg to provide RGB output.
Also I think once such conversions are available in libjpeg-turbo many projects which are maintaining color conversion routines may actually start using libjpeg-turbo for it. I think such consolidation will promote reuse and concentrated development.
But I will leave up to you to make final decision on its inclusion.
Thanks & Regards,
Mandeep
Mandeep,
YUV to RGB is not the same thing. YUV and RGB are both display colorspaces. CMYK is a print colorspace, and thus the conversion to RGB is literally undefined -- and fairly meaningless -- without a color profile. It will basically look wrong no matter what the input colorspace was, because to make it look right would require more complicated conversions.
I will buy the argument of doing the conversion internal to libjpeg, if that conversion is meant to be an intermediate step, and the RGB output will be color corrected after the fact. However, that's generally not the way color correction happens. It generally happens in the process of CMYK-to-RGB conversion. Any application that is relying on our simplified formulas is never going to produce visually correct results.
I guess my point is that I don't know if it's a good idea to encourage people to use these routines, because they don't produce correct output. It might be reasonable if we were to assume some of the most common profiles -- SWOP for CMYK and sRGB, but my understanding is that doing such a "real" color conversion is a non-linear computation.
When I look at the primary consumers of this technology, Mozilla is a big one, and it's telling that they do not do CMYK-to-RGB conversion within libjpeg. They do it external to the library. ImageMagick, likewise.
One of the posters below works for a major photo sharing site, and he's determined that it isn't useful to leverage these routines in libjpeg. I think that, at best, their usefulness is limited to providing a rough preview of the image, but even then, I don't think they're particularly useful unless they produce a rough visual approximation of the color corrected output. Currently, they don't. They produce a drastically over-saturated RGB output.
Unless you have counter-arguments, I am leaning toward not including them, simply because I don't want to have to field support tickets from people who don't understand that what we're providing isn't a true color conversion.
What would change my mind is if there were a way to make the output visually approximate the color correction process for some well-known CMYK profile. We can pre-compute the lookup tables if we need to. Such would be useful for preview purposes, and people would still be able to extract the CMYK data if they wanted to use their own conversion methods. I just don't want to go to market with something whose output doesn't look correct under any circumstances. If it only looks correct for SWOP input, then that's at least a reasonable limitation in which consumers of the technology can work.
Rejecting this for now. If application developers believe that the trivial conversion algorithm used in this patch is appropriate for their purposes, then it is very simple for them to request CMYK output from libjpeg-turbo and then do the conversion themselves. I do not believe that adding the additional CMYK-to-RGB step is appropriate without color correction, which would not be practical to implement within libjpeg-turbo. We do not want to encourage people to use an incorrect in-library conversion routine as a substitute for color correction.