Hello,

 

I would like to whitelist search engine crawlers, ideally by checking their IP range, remote host and user agent, and for matching requests, give them a pass on one or more specific rules without disabling the rule engine alltogether.

 

I'm not familiar with writing mod_security rules, so I hope I can get some help and advice here.

 

I added a file "modsecurity_crs_15_whitelist.conf" in /etc/apache2/modsecurity where all the files are parsed for rules (that can be another directory, depending on where you put your rules)

 

And I began to design a rule file for whitelisting bots.

 

Here is a non-working example

SecRule REMOTE_ADDR "^192\.168\.[0-1]{1}\.[0-9]{1,3}$" chain

SecRule REMOTE_HOST googlebot.com$ chain

SecRule REQUEST_HEADERS:User-Agent "Googlebot" phase:1,log,allow,id:999999999,ctl:ruleEngine=off

 

I want the rules to check the IP and the USER-Agent and if performance permits the Remote Host (don't know if this requires a DNS request or not).

As you see, the regex allows to check for simple ranges.

 

Some questions I would like to ask persons that are knowledgable about mod_security rules:

 

1- the above rules chain rules in an "AND" mode, i.e. if this AND that, then allow.  Question: how to introduce an OR ? i.e. if the IP address is this OR that, then allow? Would the following work?

 

SecRule REMOTE_ADDR "pm@

^192\.168\.[0-1]{1}\.[0-9]{1,3}$|^193\.168\.[0-1]{1}\.[0-9]{1,3}$|^194\.168\

.[0-1]{1}\.[0-9]{1,3}$”

 

2- I want to give matching requests a pass on one or more specific rules only, not turn off the secrule engine completely. How can this be done?

 

Thank you!