Re: [Mod-security-developers] [Vote] 2.7.5

[Rainer J. <rai...@ki...>]

On 16.07.2013 21:05, Christian Folini wrote:
> Hello Rainer,
> 
> On Tue, Jul 16, 2013 at 01:05:08PM +0200, Rainer Jung wrote:
>> Hi Christian,
>>
>> I think when running the unit tests via "make test" you should add
>> -DMSC_TEST to CFLAGS. That flag comments all httpd calls and produces
>> standalone binaries.
> 
> Thanks for the info. True, it's mentioned in the reference guide.
> It's just that I used outdated documentation.
> 
> Here we have the culprit:
> $> make CFLAGS=-DMSC_TEST test
> ...
> Loaded 8 tests from ./op/rx.t
>      1) op "rx": passed (Pattern match "" at UNIT_TEST.)
>      2) op "rx": passed
>      3) op "rx": passed (Pattern match "" at UNIT_TEST.)
>      4) op "rx": passed (Pattern match "abc" at UNIT_TEST.)
>      5) op "rx": passed (Pattern match "def" at UNIT_TEST.)
>      6) op "rx": passed (Pattern match "ghi" at UNIT_TEST.)
>      7) op "rx": passed
> ERROR: Failed to create rule for op "rx": Error creating rule: Error compiling pattern (offset 2): unrecognized character after (? or (?-
> Test exited with signal 11.
> Executed: ./msc_test "-t" "op" "-n" "rx" "-p" "(?^i:^([^=])\s*=\s*((?:abc)+(?:def|ghi){2})$)" "-D" "0" "-r" "1"
>      8) op "rx": failed
> Passed:  7; Failed:  1
> ...
> 576/577 tests passed.
> FAIL: run-unit-tests.pl
> ========================================
> 1 of 1 test failed
> Please report to su...@mo...
> ========================================
> make[2]: *** [check-TESTS] Error 1
> make[2]: Leaving directory `/usr/src/modsecurity/ModSecurity-remotes-2.7.x/tests'
> make[1]: *** [check-am] Error 2
> make[1]: Leaving directory `/usr/src/modsecurity/ModSecurity-remotes-2.7.x/tests'
> make: *** [check-recursive] Error 1

I was curious why this didn't fail for me. I vaguely remember other
users had reported the same problem before. So here's what I think is
the explanation:

The original pattern in the unit test is in op/rx.t:

    qr/^([^=])\s*=\s*((?:abc)+(?:def|ghi){2})$/i

The pattern is read by the perl script run-unit-tests.pl. Before Perl
5.14 the qr was converted to:

(?i-xsm:^([^=])\s*=\s*((?:abc)+(?:def|ghi){2})$)

Starting with Perl 5.14 there's a new syntax and the pattern results in

(?^i:^([^=])\s*=\s*((?:abc)+(?:def|ghi){2})$)

(See e.g. http://perldoc.perl.org/perlre.html#Extended-Patterns, there
look for "Starting in Perl 5.14").

Now the unit tests feed the internal representation to the compiled
msc_test binary, which calls the PCRE library to handle the regexp.
Unfortunately the PCRE library only accepts the old way of expressing
this pattern, not the new ?^ syntax.

So if you run the unit tests with a Perl before 5.14 they succeed, with
a newer Perl they fail, because a Perl pre-compiled pattern might no
longer be compatible with PCRE (despite what PCRE means).

I didn't find a bug report for the missing ?^ syntax in the PCRE bug
tracker, but "?^" is not the best token to start a search with.

Regards,

Rainer