#332 gm convert for some inputs takes a long time and then corrupts the output

[score_under]

Attached is one of the images which triggered this for us.

"gm convert input.png png8:output.png" will output a corrupt image.

"gm convert input.png +dither -colors 256 png8:output.png" will output a corrupt image, AND take a long time to do so.

[Bob Friesenhahn]

I am able to reproduce the problem here

freddy:~% gm convert scrshot_5716_2_0_20160122103651_1_None.png png8:output.png
freddy:~% gm display output.png
gm display: n[B7][A6][DB]: invalid chunk type (output.png).
freddy:~% pngcheck output.png
output.png  CRC error in chunk tRNS (computed 43480ee8, expected 84960440)
ERROR: output.png

[Glenn Randers-Pehrson]

Maybe you're running a buggy libpng. I see the same behavior on Ubuntu which
is apparently using a recently-patched libpng-1.2.50.

lrwxrwxrwx 1 root root 18 Dec 18 10:16 /lib/x86_64-linux-gnu/libpng12.so.0 -> libpng12.so.0.50.0

It would have been better to replace libpng-1.2.50 with the current libpng-1.2.56,
or better yet, rebuild with libpng-1.6.21. The bug was introduced in libpng-1.2.54beta
and almost immediately fixed, but someone could have botched the patch of libpng-1.2.50.

However, ImageMagick built with the same patched libpng-1.2.50 writes a good PNG.

gm won't run for me on freddy so I wasn't able to test there:
freddy> ldd /usr/local/bin/gm | grep png
libpng15.so.15 => /home/glennrp/freddy/lib/libpng15.so.15
libpng15.so.15 (PNG15_0) => (version not found)

[Bob Friesenhahn]

On Fri, 22 Jan 2016, Glenn Randers-Pehrson wrote:

Maybe you're running a buggy libpng. I see the same behavior on Ubuntu which is apparently using a
recently-patched libpng-1.2.50.

I see the problem with libpng 1.5.21, but not with 1.2.50, 1.4.12 or
1.6.18. However, the 1.2.50 (on a Ubuntu Linux system) may not qualify
as "recently patched".

Bob

[Bob Friesenhahn]

On Sat, 23 Jan 2016, Bob Friesenhahn wrote:

On Fri, 22 Jan 2016, Glenn Randers-Pehrson wrote:

  Maybe you're running a buggy libpng. I see the same behavior on Ubuntu which is apparently using a
  recently-patched libpng-1.2.50.

I see the problem with libpng 1.5.21, but not with 1.2.50, 1.4.12 or 1.6.18. However, the 1.2.50 (on a Ubuntu
Linux system) may not qualify as "recently patched".

I updated the system with libpng 1.5.21 to 1.5.26 and the problem went
away.

--
Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/

[Bob Friesenhahn]

Testing shows that this problem is due to an issue in libpng. Glenn says that this is due to a known bug which has since been fixed. Versions provided by OS distributions (e.g. 1.2.50) might exhibit the bug even if the claimed PNG version does not normally exhibit the bug due to OS distribution patches.

[score_under]

I've reproduced this against GraphicsMagick 1.3.23 built against libpng 1.6.20 (as well as the original on my centos machine, not sure which version that's built against but likely 1.2.49), so I don't think it's an issue with libpng being out of date. Is it some ubuntu-specific patch which fixes it?

[Bob Friesenhahn]

I definitely see the "long time to do so". For some reason, disabling dithering has always taken longer than when dithering is enabled. I have never studied to find out why. Perhaps it searches harder for the closest color since it it is not allowed to approximate via dithering.

I ran the suggested command line with the provided PNG input file under valgrind and did not see any issue other than a small memory leak which seems like part of the pthreads implemenation, and would normally be masked out by a valgrind suppression.

The origin of the problem is not clear to me. I definitely saw it go away on the system where I reproduced it by installing the latest libpng on the same release branch. While there is a libpng 1.6.21, I don't see anything in its release notes which suggests that it would solve a problem like this.

[Bob Friesenhahn]

Re-opening to provide more time to get to the bottom of this issue.

[Glenn Randers-Pehrson]

The bug was fixed in libpng shortly after it was introduced,
but evidently Ubuntu has missed this patch when patching their
libpng-1.2.50.

This introduced the bug:
version 1.2.54rc02 [October 31, 2015]
Prevent writing over-length PLTE chunk (Cosmin Truta).
Silently truncate over-length PLTE chunk while reading.

This fixed it:
version 1.2.54rc04 [November 5, 2015]
Fixed new bug with CRC error after reading an over-length palette
(bug report by Cosmin Truta) (CVE-2015-8126).

So, although I haven't tried to access Ubuntu's sources, I think they
applied the October 31 patch to their 1.2.50 (ntroducing the bug)
but failed to apply the November 5 patch that fixes the bug.

[Bob Friesenhahn]

Is the problem known to be in 1.6.20 as well? This latest apparent finding is why I re-opened the report.

[Glenn Randers-Pehrson]

It existed for the same six days in libpng-1.6.19rc02-rc04 (and in libpng-1.5.24rc01-rc04). It's not in 1.6.20 but the reporter might not be actually running with that; they need to do

gm convert -list format | grep PNG

to make sure.

[score_under]

Apologies for getting back to this issue so late. This is the output of that command on my machine:

      PNG P  rw-  Portable Network Graphics (libpng 1.6.20,1.6.21, zlib 1.2.8)
            See http://www.libpng.org/ for information on PNG..
    PNG00 P  rw-  PNG that inherits type and depth from original (libpng 1.6.20,1.6.21, zlib 1.2.8)
    PNG24 P  rw-  24-bit RGB PNG, opaque only (libpng 1.6.20,1.6.21, zlib 1.2.8)
    PNG32 P  rw-  32-bit RGBA PNG, semitransparency OK (libpng 1.6.20,1.6.21, zlib 1.2.8)
    PNG48 P  rw-  opaque or binary transparent 48-bit RGB (libpng 1.6.20,1.6.21, zlib 1.2.8)
    PNG64 P  rw-  opaque or transparent 64-bit RGBA (libpng 1.6.20,1.6.21, zlib 1.2.8)
     PNG8 P  rw-  8-bit indexed PNG, binary transparency only (libpng 1.6.20,1.6.21, zlib 1.2.8)

On two other affected machines, it is reporting these versions:

      PNG P  rw-  Portable Network Graphics (libpng 1.2.49, zlib 1.2.3)
            See http://www.libpng.org/ for information on PNG..
    PNG00 P  rw-  PNG that inherits type and depth from original (libpng 1.2.49, zlib 1.2.3)
    PNG24 P  rw-  24-bit RGB PNG, opaque only (libpng 1.2.49, zlib 1.2.3)
    PNG32 P  rw-  32-bit RGBA PNG, semitransparency OK (libpng 1.2.49, zlib 1.2.3)
    PNG48 P  rw-  opaque or binary transparent 48-bit RGB (libpng 1.2.49, zlib 1.2.3)
    PNG64 P  rw-  opaque or transparent 64-bit RGBA (libpng 1.2.49, zlib 1.2.3)
     PNG8 P  rw-  8-bit indexed PNG, binary transparency only (libpng 1.2.49, zlib 1.2.3)

      PNG P  rw-  Portable Network Graphics (libpng 1.6.19,1.6.21, zlib 1.2.8)
            See http://www.libpng.org/ for information on PNG..
    PNG00 P  rw-  PNG that inherits type and depth from original (libpng 1.6.19,1.6.21, zlib 1.2.8)
    PNG24 P  rw-  24-bit RGB PNG, opaque only (libpng 1.6.19,1.6.21, zlib 1.2.8)
    PNG32 P  rw-  32-bit RGBA PNG, semitransparency OK (libpng 1.6.19,1.6.21, zlib 1.2.8)
    PNG48 P  rw-  opaque or binary transparent 48-bit RGB (libpng 1.6.19,1.6.21, zlib 1.2.8)
    PNG64 P  rw-  opaque or transparent 64-bit RGBA (libpng 1.6.19,1.6.21, zlib 1.2.8)
     PNG8 P  rw-  8-bit indexed PNG, binary transparency only (libpng 1.6.19,1.6.21, zlib 1.2.8)