mod-security-users

[mod-security-users] Create rule to manually manipulate a collection

[Paul B. <pau...@ou...>]

I'm currently trying to work out if there is a way I can manually manipulate a collection, and also use this to enable two or more Apache servers to interact with each other, so that IP deny lists could be synchronised with each other.

I'm starting from looking at the CRS Denial-of-service rule. I'd like to be able to manually add or remove an IP from the list.

My current thoughts were to use some administrative URL's (appropriately authenticated / protected):
  <protocol://<host>/ip/block/<ip>
  <protocol://<host>/ip/unblock/<ip>

And have Modsecurty rules capture the IP out of the URL (or from a POST parameter), and use this to manipulate the IP collection.

I've been trying unsucesfully to find any examples of doing something like this. There seems to be two main challenges, which I'm not sure if there's a way to achieve:
  1) Using ModSecurity (or other apache module) capture the IP from the URL (or another element eg. POST parameter of the request)
  2) Use captured IP value to manipulate ip.dos_block - (using captured value rather than the requesting IP)


I'm assuming that to create the syncrhonisation between two or more servers I'd then have to create a rule using lua to GET/POST request to the other servers.

Any help would be greatly appreciated.
Thanks,
Paul

Re: [mod-security-users] Create rule to manually manipulate a collection

[Christian F. <chr...@ne...>]

Hey Paul,

The code that handles the collections stored on disk is known to be fairly
brittle. I am quite sure you can manipulate the collections via script - but 
I would be surprised if Apache / ModSec could handle the results without 
segfaults or other bumps. I mean this is stored in memory and casually writing
to disk. So if you change the disk, it's either overwritten, or Apache bumps
into you locking the file, whatever. I do not think it is going to read the
file anew during normal operation.

As this is about IP blacklisting, it's probably performance relevant. If it's
not, then branch out to a lua routine and have lua + db handle the stuff.
If it is performance relevant, I think you should handle it in front of
ModSec.

What I have built before is ModSec->Log->Fail2Ban. But it is relatively simple
to do ModSec->Log->TransferLogTo2ndHost->Fail2Ban. Alternatively, do
ModSec->Log->TransferLogToNetworkFirewall->BanOnFirewall.

Just my 2 cents,

Christian


On Tue, Oct 20, 2020 at 01:02:21PM +0000, Paul Beckett wrote:
> I'm currently trying to work out if there is a way I can manually manipulate
> a collection, and also use this to enable two or more Apache servers to
> interact with each other, so that IP deny lists could be synchronised with
> each other.
> 
> I'm starting from looking at the CRS Denial-of-service rule. I'd like to be
> able to manually add or remove an IP from the list.
> 
> My current thoughts were to use some administrative URL's (appropriately
> authenticated / protected): <protocol://<host>/ip/block/<ip>
> <protocol://<host>/ip/unblock/<ip>
> 
> And have Modsecurty rules capture the IP out of the URL (or from a POST
> parameter), and use this to manipulate the IP collection.
> 
> I've been trying unsucesfully to find any examples of doing something like
> this. There seems to be two main challenges, which I'm not sure if there's a
> way to achieve: 1) Using ModSecurity (or other apache module) capture the IP
> from the URL (or another element eg. POST parameter of the request) 2) Use
> captured IP value to manipulate ip.dos_block - (using captured value rather
> than the requesting IP)
> 
> 
> I'm assuming that to create the syncrhonisation between two or more servers
> I'd then have to create a rule using lua to GET/POST request to the other
> servers.
> 
> Any help would be greatly appreciated.  Thanks, Paul


> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/

Re: [mod-security-users] Create rule to manually manipulate a collection

[Paul B. <pau...@ou...>]

Christian,
Many thanks for your comments, really appreciate it.

I agree with your suggestion that for DOS protection moving this upstream and using Fail2Ban / network firewall makes more sense.

I have some other use cases which aren't suited to outright banning though. After about a day of searching and trying several different things (trying to reinitialise IP which I've discovered is impossible, or trying to find a way to modify it from lua..... none of which worked).... I stumbled across this: https://www.codeproject.com/Articles/574935/BlockplusIPplususingplusModSecurity . Essentially it's putting a SecRule before everything else that will conditionally initcol IP to be ARGS:ip. As long as that doesn't match it gets initialised by a later rule to REMOTE_IP. Some quick testing seems to suggest it works, and can be easily adapted to my needs.

Can you see any problems with this approach?

Thanks,
Paul

________________________________
From: Christian Folini <chr...@ne...>
Sent: 21 October 2020 07:30
To: mod...@li... <mod...@li...>
Subject: Re: [mod-security-users] Create rule to manually manipulate a collection

Hey Paul,

The code that handles the collections stored on disk is known to be fairly
brittle. I am quite sure you can manipulate the collections via script - but
I would be surprised if Apache / ModSec could handle the results without
segfaults or other bumps. I mean this is stored in memory and casually writing
to disk. So if you change the disk, it's either overwritten, or Apache bumps
into you locking the file, whatever. I do not think it is going to read the
file anew during normal operation.

As this is about IP blacklisting, it's probably performance relevant. If it's
not, then branch out to a lua routine and have lua + db handle the stuff.
If it is performance relevant, I think you should handle it in front of
ModSec.

What I have built before is ModSec->Log->Fail2Ban. But it is relatively simple
to do ModSec->Log->TransferLogTo2ndHost->Fail2Ban. Alternatively, do
ModSec->Log->TransferLogToNetworkFirewall->BanOnFirewall.

Just my 2 cents,

Christian


On Tue, Oct 20, 2020 at 01:02:21PM +0000, Paul Beckett wrote:
> I'm currently trying to work out if there is a way I can manually manipulate
> a collection, and also use this to enable two or more Apache servers to
> interact with each other, so that IP deny lists could be synchronised with
> each other.
>
> I'm starting from looking at the CRS Denial-of-service rule. I'd like to be
> able to manually add or remove an IP from the list.
>
> My current thoughts were to use some administrative URL's (appropriately
> authenticated / protected): <protocol://<host>/ip/block/<ip>
> <protocol://<host>/ip/unblock/<ip>
>
> And have Modsecurty rules capture the IP out of the URL (or from a POST
> parameter), and use this to manipulate the IP collection.
>
> I've been trying unsucesfully to find any examples of doing something like
> this. There seems to be two main challenges, which I'm not sure if there's a
> way to achieve: 1) Using ModSecurity (or other apache module) capture the IP
> from the URL (or another element eg. POST parameter of the request) 2) Use
> captured IP value to manipulate ip.dos_block - (using captured value rather
> than the requesting IP)
>
>
> I'm assuming that to create the syncrhonisation between two or more servers
> I'd then have to create a rule using lua to GET/POST request to the other
> servers.
>
> Any help would be greatly appreciated.  Thanks, Paul


> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/



_______________________________________________
mod-security-users mailing list
mod...@li...
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/

Re: [mod-security-users] Create rule to manually manipulate a collection

[Christian F. <chr...@ne...>]

Thanks for that link Paul.

The idea seems to be that you ban / unban IPs based on a curl request from
localhost and then the IP collection. That is quite neat and I had not thought
of that. If you can then trigger these requests instead from localhost from
the twin server, then this is likely to work - unless you are under a DoS.

Best,

Christian

On Wed, Oct 21, 2020 at 02:38:39PM +0000, Paul Beckett wrote:
> Christian,
> Many thanks for your comments, really appreciate it.
> 
> I agree with your suggestion that for DOS protection moving this upstream and using Fail2Ban / network firewall makes more sense.
> 
> I have some other use cases which aren't suited to outright banning though. After about a day of searching and trying several different things (trying to reinitialise IP which I've discovered is impossible, or trying to find a way to modify it from lua..... none of which worked).... I stumbled across this: https://www.codeproject.com/Articles/574935/BlockplusIPplususingplusModSecurity . Essentially it's putting a SecRule before everything else that will conditionally initcol IP to be ARGS:ip. As long as that doesn't match it gets initialised by a later rule to REMOTE_IP. Some quick testing seems to suggest it works, and can be easily adapted to my needs.
> 
> Can you see any problems with this approach?
> 
> Thanks,
> Paul
> 
> ________________________________
> From: Christian Folini <chr...@ne...>
> Sent: 21 October 2020 07:30
> To: mod...@li... <mod...@li...>
> Subject: Re: [mod-security-users] Create rule to manually manipulate a collection
> 
> Hey Paul,
> 
> The code that handles the collections stored on disk is known to be fairly
> brittle. I am quite sure you can manipulate the collections via script - but
> I would be surprised if Apache / ModSec could handle the results without
> segfaults or other bumps. I mean this is stored in memory and casually writing
> to disk. So if you change the disk, it's either overwritten, or Apache bumps
> into you locking the file, whatever. I do not think it is going to read the
> file anew during normal operation.
> 
> As this is about IP blacklisting, it's probably performance relevant. If it's
> not, then branch out to a lua routine and have lua + db handle the stuff.
> If it is performance relevant, I think you should handle it in front of
> ModSec.
> 
> What I have built before is ModSec->Log->Fail2Ban. But it is relatively simple
> to do ModSec->Log->TransferLogTo2ndHost->Fail2Ban. Alternatively, do
> ModSec->Log->TransferLogToNetworkFirewall->BanOnFirewall.
> 
> Just my 2 cents,
> 
> Christian
> 
> 
> On Tue, Oct 20, 2020 at 01:02:21PM +0000, Paul Beckett wrote:
> > I'm currently trying to work out if there is a way I can manually manipulate
> > a collection, and also use this to enable two or more Apache servers to
> > interact with each other, so that IP deny lists could be synchronised with
> > each other.
> >
> > I'm starting from looking at the CRS Denial-of-service rule. I'd like to be
> > able to manually add or remove an IP from the list.
> >
> > My current thoughts were to use some administrative URL's (appropriately
> > authenticated / protected): <protocol://<host>/ip/block/<ip>
> > <protocol://<host>/ip/unblock/<ip>
> >
> > And have Modsecurty rules capture the IP out of the URL (or from a POST
> > parameter), and use this to manipulate the IP collection.
> >
> > I've been trying unsucesfully to find any examples of doing something like
> > this. There seems to be two main challenges, which I'm not sure if there's a
> > way to achieve: 1) Using ModSecurity (or other apache module) capture the IP
> > from the URL (or another element eg. POST parameter of the request) 2) Use
> > captured IP value to manipulate ip.dos_block - (using captured value rather
> > than the requesting IP)
> >
> >
> > I'm assuming that to create the syncrhonisation between two or more servers
> > I'd then have to create a rule using lua to GET/POST request to the other
> > servers.
> >
> > Any help would be greatly appreciated.  Thanks, Paul
> 
> 
> > _______________________________________________
> > mod-security-users mailing list
> > mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > http://www.modsecurity.org/projects/commercial/rules/
> > http://www.modsecurity.org/projects/commercial/support/
> 
> 
> 
> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/


> _______________________________________________
> mod-security-users mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/