#169 Blank response when PCRE backtrack limit is reached


Using the latest stable GeSHi release (tested with both and, highlighting the following string as JavaScript fails: http://paste.pocoo.org/raw/247939/

This does not throw an error (even if error_reporting is on), but just returns an empty response; not even a single HTTP header is being sent out! Looks like a serious bug, but I have no idea how or where it happens.

I'm running PHP 5.2.6 on Linux 2.6.26-2-686. My PCRE configuration is the default; here are the settings again (taken from phpinfo()):

pcre.backtrack_limit: 100000
pcre.recursion_limit: 100000

It seems to be a PCRE bug where the limit backtrack limit is reached.

I also tested this using the latest bleeding edge version of GeSHi and that managed to highlight this piece of code just fine, without any errors or blank responses. I'm considering switching to this dev version now. The only thing holding me back is that there’s no equivalent of GeSHi::enable_classes() in GeSHi 1.1.2alpha5 yet…

Here's a chat log from the #geshi IRC channel regarding this bug:

[12:26] <matjas> The bug appears to be fixed in GeSHi 1.1.2alpha5 though, so I was thinking of switching. But alas, I'll just wait until there's an enableClasses() method
[12:26] <BenBE> I always batch-process the SF.net bugtracker items, thus responses there take a bit longer ...
[12:28] <BenBE> http://benbe.home.omorphia.de:43815/geshi-misc/profiling/geshi-trunk/contrib/example.php --> I don't see a highlighting issue there.
[12:28] <matjas> BenBE: Please let me know if there's anything I can do to help
[12:29] <matjas> BenBE: I've created an example page here: http://jsperf.com/geshi-bug.php
[12:29] <matjas> http://web-sniffer.net/?url=http://jsperf.com/geshi-bug.php
[12:29] <matjas> I'll post the PHP source in a sec…
[12:29] <BenBE> But I guess I know where the blank response issue comes from.
[12:29] <matjas> PHP: http://paste.pocoo.org/raw/248360/
[12:30] <BenBE> please disable highlighting of numbers for a try.
[12:30] <matjas> How?
[12:30] <BenBE> This internally uses the PCRE library which had shown some issues like this for keyword highlighting and some other instances where it's used a bit more excessively in the past.
[12:31] <matjas> Oh, so it's a PCRE bug
[12:31] <BenBE> There are two ways: GeSHi: enable_numbers IIRC (see the 1.0.X docs) or inside the language file via PARSER_CONTROL --> ENABLE_FLAGS --> NUMBERS => GESHI_NEVER
[12:31] <matjas> Glad to see there’s a workaround in the GeSHi dev trunk
[12:32] <matjas> I can only find a enable_line_numbers() method in the 1.0.X docs
[12:32] <BenBE> Well, the GeSHi dev trunk uses basically the same techniques as the stable branch ...
[12:33] <BenBE> But the parser is a full rewrite so it seems the bug doesn't manifest there.
[12:33] <matjas> I can't seem to find PARSER_CONTROL in the language file either :/ I must be doing something wrong here…
[12:34] <BenBE> Another try (with enabled number highlighting): Seta small /**/ comment inside your array to split the array into smaller peaces of code ...
[12:34] <BenBE> I guess GeSHi reaches the backtracing limit with this Regexp there.
[12:34] <BenBE> PARSER_CONTROL is an optional block.
[12:35] <matjas> BenBE: http://jsperf.com/geshi-bug.php Yup, that seems to work. So you're right about the backtracing limit
[12:35] <BenBE> It's rarely documented as I usually don't want to get language files where people rely on it.
[12:36] <BenBE> k, then this will get a bit messy ...
[12:36] <BenBE> Do you know PCRE quite well?
[12:36] <matjas> Not at all, actually :)
[12:36] <BenBE> That's bad ...
[12:36] <BenBE> Because I can't reproduce it locally and thus can't test ...
[12:37] <BenBE> Could you test with the qbnz.com demo site if it's reproducable there?
[12:37] <matjas> Perhaps I can check PCRE config/settings or something
[12:37] <matjas> Ok, sure!
[12:39] <BenBE> Well, please add the current findings to your bug report, including your OS and PHP version.
[12:39] <matjas> It seems to work on http://qbnz.com/highlighter/demo.php
[12:39] <BenBE> PHP 5.2.1 on Win2K doesn't have that one (well, it's more robust than more recent versions; thou I basically just didn't have the time to update yet)
[12:40] <BenBE> That's PHP 5.3.X on Debian.
[12:40] <BenBE> Default settings for PCRE.


  • BenBE

    BenBE - 2010-08-10

    Well, most stuff was already told in the chatlog, but here some details:

    Inside geshi.php there's an array with PCRE expressions that are used for number highlighting. A fix there would include to rewrite the expressions there to avoid backtracking as much as possible. But as I can't reproduce the original issue I can't really check if this will successfully stop the issue from showing - the most you can do is increase the boundaries when it's showing.

    For a temporary workaround trying to minimize non-string block size (e.g. by inserting comments or strings) seems to help.

  • BenBE

    BenBE - 2010-08-10
    • assigned_to: nobody --> benbe
    • priority: 5 --> 3
    • milestone: --> Next_Release_(Stable)
    • labels: --> General Bugs
    • status: open --> open-accepted

Log in to post a comment.