#14 pdf file with extra carriage returns not recognized

[Gary Peck]

I have an encrypted PDF file (I do not know the software used to generate it unfortunately) that was not being recognized by qpdf:

$ qpdf --check foo.pdf
foo.pdf: not a PDF file

I examined the file's raw contents and noticed that the %PDF-1.3 header was preceded by 11 carriage returns (i.e. 11 pairs of 0D0A bytes). After manually removing these 22 bytes from the beginning of the file, qpdf was able to process and decrypt it properly.

It'd be good to handle these PDF files by automatically stripping leading carriage returns.

Thanks,
Gary

[Jay Berkenbilt]

Thanks for your report. Given that the Adobe PDF specification's implementation notes indicate that Adobe Reader recognizes a PDF file whose header appears within the first 1024 bytes and that several other PDF readers seem to accept this, I believe this is a very reasonable request. I will add it to the to-do list for the next release of qpdf, whenever that is. Thanks!

[Jay Berkenbilt]

Would it be possible for you to post the file that you originally reported this bug about? I'm curious as to whether the cross reference table in the file contains offsets from the actual beginning of the file or offsets from the header where it actually appears. Thanks.

[Gary Peck]

Sorry, but the file contains private data I can't share. I'm not familiar with the PDF format, but if you want to give me some instructions on how to locate and decode the cross-reference table, I can probably get the information you're looking for out of the file. I'm an experienced programmer, so technical instructions, pseudo-code, etc. are fine.

[Jay Berkenbilt]

I'm sorry for the long delay in response. Realistically, I'm not really going to have time to provide instructions, even to a fellow programmer, on how to do the analysis, so I'm going to go ahead and close this bug. I do have a note in my to-do list to take care of this eventually, but until I have an actual test file from "the wild", I don't have any intention to try to fix it. I'd just be guessing at that point. Thanks.

[Gary Peck]

No problem. I understand the difficulty. If I come across another example that I can share, I'll let you know.

[Anonymous]

Hello,

I have a problem with legitimate PDF files that qpdf 2.3.1 on debian squeeze fails to decrypt. It says "not a PDF file". But it is.

The Raw files indicate a leading "." (0A in hexa) before the %PDF header, acrobat reader seems not to care, but qpdf does.

After removing the leading ".", qpdf recognized the file properly.

I have attached said file. Is it related to the bug mentioned above? Was is fixed on later releases?

Thanks.

[Jay Berkenbilt]

Thanks for sharing this sample file. With this file, I should be able to fix the problem. I am planning on taking some time off work during the last week of the year to do a qpdf update. This will be on the list.

[Jay Berkenbilt]

The next release will either be 3.0.3 or 3.1.0 depending on which features I have time to implement. I am not planning on backporting this to the 2.x series.

[Jay Berkenbilt]

I have implemented a fix for this. It will be in the next release which I plan to get out this week. The change is on github now.

[Jay Berkenbilt]

Since I had to make incompatible changes, the next release will be 4.0.0. This is going out today and will include a fix for this issue.

[Anonymous]

It seems that the search for the header doesn't cover a PDF like the one here: https://www.surrey.ac.uk/files/pdf/CHAPTER%206.PDF.htm .
I've run in to other PDFs with a header much like the one I'm linking to, but I'm not sure where the binary stuff before the PDF header actually comes from.
Did the following on a Mac:

% wget https://www.surrey.ac.uk/files/pdf/CHAPTER%206.PDF.htm -O /tmp/chapter6.pdf
--2017-06-28 16:08:59--  https://www.surrey.ac.uk/files/pdf/CHAPTER%206.PDF.htm
Resolving www.surrey.ac.uk... 131.227.132.127
Connecting to www.surrey.ac.uk|131.227.132.127|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1507072 (1.4M) [text/html]
Saving to: ‘/tmp/chapter6.pdf’

/tmp/chapter6.pdf                                  100%[================================================================================================================>]   1.44M  6.06MB/s    in 0.2s    

2017-06-28 16:08:59 (6.06 MB/s) - ‘/tmp/chapter6.pdf’ saved [1507072/1507072]

% qpdf --check /tmp/chapter6.pdf 
/tmp/chapter6.pdf: not a PDF file
% qpdf --version
qpdf version 6.0.0
Copyright (c) 2005-2015 Jay Berkenbilt
This software may be distributed under the terms of version 2 of the
Artistic License which may be found in the source distribution.  It is
provided "as is" without express or implied warranty.

[Jay Berkenbilt]

I've been making some updates over the last few days. I haven't released anything yet, but qpdf built from the head of master on github (https://github.com/qpdf/qpdf) can actually handle this file now. I've made many enhancements to qpdf's ability to recover from all sorts of broken files. Do you have a way of testing that? I will probably make a 7.0.0 alpha release at some point in the near future.

[Anonymous]

Thanks for looking at this, Jay!
I have not tried to debug this or anything -- I just assumed that the problem is that the line
PCRE::Match m1 = header_re.match(line.c_str());
will fail on PDFs that have binary null bytes before the header. I don't know anything about the PCRE syntax, but to me it looks like the expression in the code will actually match if only the complete buffer is was passed on to the PCRE library (by using line.data() and line.size() instead of just line.c_str()).
I assume from a quick look at the new code, that the fixes you've made will cause qpdf to issue a warning (and then continue as if the file had specified PDF version 1.2), right? This is better than failing, but I think that actually matching the header would be even better. :-)

[Jay Berkenbilt]

As it happens, I just finished removing the dependency on libpcre from qpdf. I am still running tests and haven't pushed to master yet, but I will today or tomorrow. The new header detection code should be much more resilient in finding the header with nulls preceding it, so I doubt there will be an issue.

By the way, I'm using issue tracking at https://github.com/qpdf/qpdf now rather than sourceforge. If you find any new issues, it would be easier if you create them there. I don't think I was yet doing that back in 2011 when this was opened. :-)