#539 Images with libjpeg warnings result in error

[Matthias Kiefer]

Since release 1.3.27 GraphicsMagick treats (some) JPEG warnings as errors. However, there are a lot of images out there that produce for example the warning ‘Invalid SOS parameters for sequential JPEG’ but still seem to be valid JPEGs. At least GraphicsMagick 1.3.26 and all other tools I tried are able to display these images.

I found the following devices create JPEGs resulting in this warning and can no longer be loaded by GraphicsMagick:
- Samsung Galaxy S6, S7, S8+ with Android 7 (at least front camera and panorama)
- Samsung Galaxy A3 with Android 7 (at least the back camera)

For your reference I attached one of such images taken with a Samsung Galaxy S7 with Android 7.

Please adapt GraphicsMagick so that it can be used again to convert these images.

[Bob Friesenhahn]

On Thu, 11 Jan 2018, Matthias Kiefer wrote:

For your reference I attached one of such images taken with a Samsung Galaxy S7 with Android 7.

Please attach your sample JPEG in a zip file or some other common
archiving mechanism. SourceForge re-writes/optimizes all web file
formats (e.g GIF, JPEG, PNG) when the files are uploaded to the
tracking system so the file will have lost the original issue since it
is not the same file.

Bob

[Matthias Kiefer]

Attached you can find the zipped image.

[Matthias Kiefer]

In addition to the warning 'Invalid SOS parameters for sequential JPEG', I also found that a lot of images that result in warnings starting with "Corrupt JPEG data:" are now reported as error but have been successfully been converted before with GraphicsMagick 1.3.26.

If you want I can also try to identify some devices that these images have been taken with.

[Bob Friesenhahn]

On Thu, 11 Jan 2018, Matthias Kiefer wrote:

In addition to the warning 'Invalid SOS parameters for sequential JPEG', I also found that a lot of images that result in warnings starting with "Corrupt JPEG data:" are now reported as error but have been successfully been converted before with GraphicsMagick 1.3.26.

The cause of these issues is this change:

2017-07-08 Bob Friesenhahn bfriesen@simple.dallas.tx.us

     * coders/png.c (ReadOneJNGImage): Fix double-frees caused by
     commit on 2017-07-06.

     * coders/jpeg.c (ReadJPEGImage): Defer creating pixel cache until
     after successfully reading first scanline.  Classify some serious
     libjpeg reported "warnings" as errors and quit processing
     scanlines immediately upon first error so that corrupt JPEG does
     not consume excessive resources.  Resolves excessive resource
     consumption issue reported for two JPEG files provided via email
     by LCatro on Tue, 4 Jul 2017.

The reason for the change was that some sample files were provided
which almost took over the computer (memory and disk) and for which
libjpeg only reported a warning (likely thousands of times) as it
churned through an intentionally defective JPEG file. The change is a
band-aid on a problem which is really in "libjpeg" (and there are now
several "libjpeg"s of different maintainership and vintages) itself.

Bob

[Matthias Kiefer]

I understand the reason for this. However, from my point of view there are a lot of images out there that trigger these warnings but can nevertheless be processed.
E.g. I took a random sample from some of our service instances that generate previews from images that people upload. In this sample there have been about 10k images that triggered the warning with the invalid SOS parameters and 2k images that triggered one of the Corrupt JPEG data warnings. All of them have been successfully processed by GraphicsMagick 1.3.26.

Handling these warnings as error unconditionally, makes GraphicsMagick too restrictive in my opinion. Maybe there is better way to handle the problematic samples? Or maybe provide the possibility to disable this restrictive handling of warnings?

Matthias

[Bob Friesenhahn]

On Fri, 12 Jan 2018, Matthias Kiefer wrote:

I understand the reason for this. However, from my point of view there are a lot of images out there that trigger these warnings but can nevertheless be processed.
E.g. I took a random sample from some of our service instances that generate previews from images that people upload. In this sample there have been about 10k images that triggered the warning with the invalid SOS parameters and 2k images that triggered one of the Corrupt JPEG data warnings. All of them have been successfully processed by GraphicsMagick 1.3.26.

Handling these warnings as error unconditionally, makes
GraphicsMagick too restrictive in my opinion. Maybe there is better
way to handle the problematic samples? Or maybe provide the
possibility to disable this restrictive handling of warnings?

I will make sure to address this in the next release. There is often
a trade-off between security and usability.

The restrictive handling was added in Mercurial changeset
15063:b4139088b49a. Part of the changeset should remain, but it seems
that the mapping of some warnings to errors was too restrictive.

We continually make the development source code available via
Mercurial and development snapshots but it seems that there is no
enough independent testing to assure that problems are not added in
the release.

It may be that JWRN_JPEG_EOF "Premature end of JPEG file" should still
be treated as an error. Are you getting many "Premature end of JPEG
file" errors?

For test cases we were provided, libjpeg continued to plow ahead
"reading" non-existing data although it had already hit the end of the
file. With a crafted file, it might do this for an hour. It is a
long time since the days of dial-up modems when sometimes files were
not transferred completely.

Bob

[Bob Friesenhahn]

I will await your response as to if many of the JPEG files which can
not be read are due to "Premature end of JPEG file". If this
condition is not common/normal in modern times, I would leave the
remapping from warning to error in place at this time for truncated
files.

Bob

[Matthias Kiefer]

Thanks for addressing this!

In the sample mentioned above I see 68 warnings "Premature end of JPEG file" which however were all still successfully processed. BTW: we are using libjpeg62-turbo.

Let me know, if you need more information.

[Bob Friesenhahn]

I have files here which warn "Corrupt JPEG data: premature end of data segment" and then somehow continue on "reading" a 65281x65496 image so they are "successfully processed". Without applying resource limits, this is harmful to the computer. How do you know if the files which reported "Premature end of JPEG file" were "successfully processed" (returned a proper image) vs libjpeg just making something up such as a region of black pixels?

It is a problem what libjpeg is not particularly helpful with identifying and rejecting problem files. Only libjpeg is in a position to know what is going on.

[Bob Friesenhahn]

The promotion of certain warnings to errors is removed by Mercurial changeset 15331:8b560f627220. This changeset also adds a rationalization for claimed image dimensions based on image size. Thank you for reporting this issue.

[Matthias Kiefer]

Thanks for taking care so fast! Is there already a plan for a release containing this changeset?

Regarding the corrupt images: I did not have a look at the result image but at least there has been no error in the further processing. Having an broken image filled up with black pixels would be acceptable as having some result is better than none.

Of course we have applied resource limits, using the MAGICK environment variables and using the resource and processing limits of the operating system. After all there are also valid files that show a similiar behaviour as you described without a warning from libjpeg. See e.g. https://bomb.codes/bombs

[Bob Friesenhahn]

If all goes well, there should be a new release by the end of the month.

[Matthias Kiefer]

Ok. Thank you very much!

Matthias