#361 Hang on startup since commit #ab767c

[Thomas Zajic]

Since commit #ab767c, which must have been roughly around the time when 2.6.0 was released IIRC, gscan2pdf wouldn't start up properly for me anymore. It would only show a blank window without any GUI elements whatsoever, hanging on the commandline with the following error message:

[zlatko@disclosure:~/usrlocal/src/gscan2pdf]$ ./gscan2pdf --log=file.log
INFO - Starting gscan2pdf 2.9.0
INFO - Called with perl -I /usr/local/lib/perl5/site_perl -I /usr/local/lib/perl5 -I /usr/local/lib/perl5/site_perl/i386-linux -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/i386-linux -I /usr/local/lib/perl5/site_perl/5.8.7/i486-linux -I /usr/local/lib/perl5/site_perl/5.8.6/i486-linux -I /usr/local/lib/perl5/site_perl/5.8.4/i486-linux -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/5.10.0/i486-linux-thread-multi -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/5.10.0/i486-linux-thread-multi -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/5.10.0/i486-linux-thread-multi -I /usr/local/lib/perl5/site_perl/5.10.0/i486-linux-thread-multi -I /usr/local/lib/perl5/site_perl/5.10.0/i486-linux -I /usr/local/lib/perl5/site_perl/5.10.0 -I /usr/local/lib/perl5/site_perl/5.8.8/i486-linux-thread-multi -I /usr/local/lib/perl5/site_perl/5.8.8/i486-linux -I /usr/local/lib/perl5/site_perl/5.8.8 -I /usr/local/lib/perl5/5.10.0/i486-linux-thread-multi -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/5.10.0/i486-linux-thread-multi -I /usr/local/lib/perl5/5.10.0 -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/5.10.0/i486-linux-thread-multi -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/5.10.0/i486-linux-thread-multi -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/5.10.0 -I /usr/local/share/perl5 /usr/local/src/gscan2pdf/gscan2pdf --log=/usr/local/src/gscan2pdf/file.log
INFO - Log level DEBUG
INFO - Using de_AT.UTF-8 locale
INFO - Startup LC_NUMERIC C
INFO - Reading config from /home/zlatko/.config/gscan2pdfrc
INFO - Config file version v2.9.0
[...]
INFO - convert rose: /tmp/gscan2pdf-OIHl/DJ2Rx48MXr.tif
INFO - Spawned PID 21386
WARN - *** unhandled exception in callback:
***   Negative length at /usr/lib/perl5/IO/Handle.pm line 463.
***  ignoring at /usr/local/lib/perl5/Glib/Object/Introspection.pm line 67.

^C
[zlatko@disclosure:~/usrlocal/src/gscan2pdf]$

With a few debug prints I found out the the error occurs at line 958, after calling my $imgobj = $pdfobj->image_tiff($temptif);, but I'm not Perl-fluent enough to dig any deeper. It was easier for me to create a simple patch to revert commit #ab767c instead, and have it use tiff2pdf during the dependency check like before. :-) I haven't noticed any problems since then during actual usage, generating and saving PDFs works fine as usual, it was just the dependency check on startup that didn't work.

The full file.log is attached, maybe it helps to shed some light.

And yes, I know ... sorry for my $PERLPATH from hell. :-D I have accumulated tons of modules in /usr/local from many different Perl versions included in many different Slackware releases over the years, and my $PERLPATH shows. I do however update & recompile those that make use of shared objetcs whenever I notice that they don't work anymore, so I'm pretty sure that this isn't part of the problem. Feel free to prove me wrong, though. ;-)

And last but not least: thanks for a great piece of software! :-)

[Jeffrey Ratcliffe]

Thanks for the report.

The weird thing is that the code you debugged is also used in a more complicated form when creating PDFs with LZW, Zip or Packbits compression. Perhaps you've never tried those options. Can you?

I'm guessing that the problem must be somewhere inside PDF::API2.

You can get a better backtrace with the perl debugger:

perl -d -I /usr/local/lib/perl5/site_perl -I /usr/local/lib/perl5 -I /usr/local/lib/perl5/site_perl/i386-linux -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/i386-linux -I /usr/local/lib/perl5/site_perl/5.8.7/i486-linux -I /usr/local/lib/perl5/site_perl/5.8.6/i486-linux -I /usr/local/lib/perl5/site_perl/5.8.4/i486-linux -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/5.10.0/i486-linux-thread-multi -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/5.10.0/i486-linux-thread-multi -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/5.10.0/i486-linux-thread-multi -I /usr/local/lib/perl5/site_perl/5.10.0/i486-linux-thread-multi -I /usr/local/lib/perl5/site_perl/5.10.0/i486-linux -I /usr/local/lib/perl5/site_perl/5.10.0 -I /usr/local/lib/perl5/site_perl/5.8.8/i486-linux-thread-multi -I /usr/local/lib/perl5/site_perl/5.8.8/i486-linux -I /usr/local/lib/perl5/site_perl/5.8.8 -I /usr/local/lib/perl5/5.10.0/i486-linux-thread-multi -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/5.10.0/i486-linux-thread-multi -I /usr/local/lib/perl5/5.10.0 -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/5.10.0/i486-linux-thread-multi -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/5.10.0/i486-linux-thread-multi -I /usr/local/stow/Perl-0-perllocal.pod/lib/perl5/5.10.0 -I /usr/local/share/perl5 /usr/local/src/gscan2pdf/gscan2pdf --log=/usr/local/src/gscan2pdf/file.log

It'll then prompt before starting the program. Type r to run it. When it crashes, type T for a backtrace, and then q to quit.

I'm interested how you have installed the program. /usr/local/src/gscan2pdf/gscan2pdf looks like you've dumped the source there, but in that case, there would normally be a bin directory. What is in /usr/local/src/gscan2pdf?

[Thomas Zajic]

The strange "installation" in /usr/local/src/gscan2pdf/gscan2pdf was only used for this report, I just copied the main "binary" there as I was too lazy to reverse-patch my actual installation in its original folder. Probably not a good idea, as it obviously created some confusion/irritation here. Sorry about that!

The (rest of the) actual installation lives in /usr/local/stow/gscan2pdf-2.9.0, which is then symlinked back to the /usr/local/ hierarchy using - yes, you guessed it ;-) - GNU stow. I now added the unpatched main "binary" to the normal location using gscan2pdf-unpatched as a name, and I'll use this one for further debugging from now on. The symlink indirection by stow and my $PERLPATH already cause more than enough obfuscation anyway. I'll probably also try to also cut down my $PERLPATH to contain only the things needed for gscan2pdf to get rid of unneeded cruft.

I'll report back here as soon as I found something, eiter a crash in the compression options you mentioned, or a usable backtrace from the perl -d run.

Thanks! :-)

[Thomas Zajic]

Wow, you were spot-on with your assumption that the problem is within PDF::API2. I created a minimalistic test.pl consisting only of the failing code block:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

#!/usr/bin/perl use warnings; use strict; use feature 'switch'; no if $] >= 5.018, warnings => 'experimental::smartmatch'; use PDF::API2; open(my $temptif, "<", "/tmp/test.tif"); my $temppdf = File::Temp->new( DIR => '/tmp', SUFFIX => '.pdf' ); my $pdfobj = PDF::API2->new( -file => $temppdf ); my $page = $pdfobj->page; my $size = 150; $page->mediabox( $size, $size ); my $gfx = $page->gfx; my $imgobj = $pdfobj->image_tiff($temptif); $gfx->image( $imgobj, 0, 0, $size, $size ); $pdfobj->save; $pdfobj->end;

I then created /tmp/test.tif exactly the same way it is created in gscan2pdf:

[zlatko@disclosure:~]$ convert rose: /tmp/test.tif
[zlatko@disclosure:~]$ ls -la /tmp/test.tif
-rw-r--r-- 1 zlatko users 9932 Sep 23 12:50 /tmp/test.tif
[zlatko@disclosure:~]$ file /tmp/test.tif
/tmp/test.tif: TIFF image data, little-endian, direntries=15, height=46, bps=9854, compression=none, PhotometricIntepretation=RGB, orientation=upper-left, width=70
[zlatko@disclosure:~]$ md5sum /tmp/test.tif
1c3bb060364cd2da531531906373cd38  /tmp/test.tif

Ready, set, go:

[zlatko@disclosure:~]$ perl -d test.pl            

Loading DB routines from perl5db.pl version 1.49
Editor support available.

Enter h or 'h h' for help, or 'man perldebug' for more help.

main::(test.pl:10): open(my $temptif, "<", "/tmp/test.tif");
  DB<1> r
Negative length at /usr/lib/perl5/IO/Handle.pm line 463.
 at /usr/lib/perl5/IO/Handle.pm line 463.
    IO::Handle::read(GLOB(0x8aa04d8), "", 2187478905) called at /usr/local/share/perl5/PDF/API2/Resource/XObject/Image/TIFF.pm line 126
    PDF::API2::Resource::XObject::Image::TIFF::handle_generic(PDF::API2::Resource::XObject::Image::TIFF=HASH(0x9b2e1c0), PDF::API2::Basic::PDF::File=HASH(0x9a94d80), PDF::API2::Resource::XObject::Image::TIFF::File=HASH(0x9ae6cc8)) called at /usr/local/share/perl5/PDF/API2/Resource/XObject/Image/TIFF.pm line 267
    PDF::API2::Resource::XObject::Image::TIFF::read_tiff(PDF::API2::Resource::XObject::Image::TIFF=HASH(0x9b2e1c0), PDF::API2::Basic::PDF::File=HASH(0x9a94d80), PDF::API2::Resource::XObject::Image::TIFF::File=HASH(0x9ae6cc8)) called at /usr/local/share/perl5/PDF/API2/Resource/XObject/Image/TIFF.pm line 50
    PDF::API2::Resource::XObject::Image::TIFF::new("PDF::API2::Resource::XObject::Image::TIFF", PDF::API2::Basic::PDF::File=HASH(0x9a94d80), GLOB(0x8aa04d8)) called at /usr/local/share/perl5/PDF/API2.pm line 2037
    PDF::API2::image_tiff(PDF::API2=HASH(0x8a9f928), GLOB(0x8aa04d8)) called at test.pl line 17
Debugged program terminated.  Use q to quit or R to restart,
use o inhibit_exit to avoid stopping after program termination,
h q, h R or h o to get additional info.
  DB<1> q
[zlatko@disclosure:~]$

Ho-hum. I then tested this script with a few other TIFFs I had lying around, and they all did not cause this error. I then used convert to create a JPG instead of a TIFF, and this did also not cause the error. The TIFF was also displayed fine in xv, gimp, eog, gthumb and geequie, and neither of these viewers showed any warning that something might be wrong with the image.

So the problem appears to be two-fold: (my version of) convert creates a possibly somehow out-of-specs TIFF, and PDF::API2 crashes on reading it instead of handling it gracefully. Or it hits some obscure corner case in its TIFF loading code. I stepped through it once in the debugger, and it appears to read the file in 12-byte-chunks. The size of the convert-created TIFF is not a multiple of 12 bytes. I checked a couple (but not all, only three or so) of the other TIFFs, and all of their sizes were multiples of 12 bytes.

Could this be it, something so simple and stupid? I'll attach my test.tif so you can compare it to the one that your version of convert creates. I'll try to find & dig into the TIFF specs a bit meanwhile. Possibly relevant information:

[zlatko@disclosure:~]$ convert --version
Version: ImageMagick 6.9.4-9 Q16 i586 2016-06-17 http://www.imagemagick.org
[...]
[zlatko@disclosure:~]$ tiff2pdf -h
LIBTIFF, Version 4.1.0
[...]

[Jeffrey Ratcliffe]

Hmm. I've found bugs in PDF::API2's TIFF implementation before, which is why I wrote Graphics::TIFF to allow PDF::API2 to use libtiff directly to access the images rather than reinventing the wheel. Unfortunately, the author didn't bite, despite my giving him the patches necessary.

Another guy forked PDF::API2 to PDF::Builder, and has used my patches, so the next release of gscan2pdf will probably use PDF::Builder.

Back to your problem - you are right - JPEG is much better tested than TIFF in PDF::API2. I only kept the TIFF out of laziness because I was using tiff2pdf before to create the test PDF.

I'll change to testing with a JPEG and your problem should disappear.

[Thomas Zajic]

Okay, thanks a lot. I've already seen the patches you mentioned in CPANs RT, while I was looking for bug reports against PDF::API2 which could be related to this one.

I meanwhile also tried the convert rose: test.tif thing on an FC32 workstation using a newer version of IM & convert. The test.tif it created was in fact 8 bytes smaller in size, and thus its size was a multiple of 12. This file was happily accepted by PDF::API2 and didn't cause an error.

So it looks like it really was a combination of a "slightly unusual" TIFF and PDF::API2's non-existent error handling of such a case. Oh well. Feel free to close this ticket, and thanks for your efforts! :-)

[Jeffrey Ratcliffe]

I've just committed this patch, which should fix things for you.