Hi there, no thats what ive been doing currently, i need to just need to find a way to run the script on a schedule rather than on every hit for now ? If i run the script like i did with the initial import it will add all the ones i currently have in there aswell wont it ?

let me know thanks heaps.

Ryan Barnett wrote:

If you don’t want to send your audit log data to the Console yet, you can still keep in running in concurrent mode.  Just update the SecAuditLog directive to point to a file instead of to the modsec-auditlog-collector script line.

 

--
Ryan C. Barnett
Breach Security: Director of Application Security Training
Web Application Security Consortium (WASC) Member
CIS Apache Benchmark Project Lead
SANS Instructor, GCIA, GCFA, GCIH, GSNA, GCUX, GSEC
Author: Preventing Web Attacks with Apache

 


From: Dan Rossi [mailto:spam@electroteque.org]
Sent: Friday, December 29, 2006 4:03 AM
To: Ryan Barnett
Cc: mod-security-users@lists.sourceforge.net
Subject: Re: [mod-security-users] how to get console to collect concurrent logs

 

Hi Ryan, we currently have our servers in detection mode atm we have 5 freebsd boxes to monitor. Is there a way to run the log generators manually without going via AuditLog during the period of fixing up false positives then turn it on to pipe when everything has stabilised, i feel the traffic is going to create a problem as its trigerring mod sec quite a bit.

let me know thanks.

Ryan Barnett wrote:

See comments inline below.

 

--
Ryan C. Barnett
Breach Security: Director of Application Security Training
Web Application Security Consortium (WASC) Member
CIS Apache Benchmark Project Lead
SANS Instructor, GCIA, GCFA, GCIH, GSNA, GCUX, GSEC
Author: Preventing Web Attacks with Apache

 


From: Dan Rossi [mailto:spam@electroteque.org]
Sent: Thursday, December 28, 2006 2:41 AM
To: Ryan Barnett
Cc: mod-security-users@lists.sourceforge.net
Subject: Re: [mod-security-users] how to get console to collect concurrent logs

 

Hi thanks for the clarification however im reading the mod security 2.0 docs as thats what im using.

There is some basic information in the script which is enough to work with, so it posts the data to the single hosted console !

Im a little worried about this comment though, so it will kill apache if its run in production ?

[Ryan Barnett] The modsec-auditlog-collector script may work fine for some smaller organizations.  The 2 main items that would impact its effectiveness are –

  1. The current amount of client traffic on the Apache web server.  The script can only handle one file at a time and has problems when under heavy load.
  2. How you have the modsecurity SecAuditEngine configured.  If the SecAuditEngine is set to on, it will log all traffic to the auditlog (including requests for downloading images, etc…).  This would drastically impact this scripts ability to process the INDEX file and send data to the Console host.  You should use SecAuditEngine RelevantOnly and set the SecAuditEngineRelevantStatus to something like “^(?:4|5)” so that is will only log 4XX and 5XX level status code transactions to the auditlog.

 

Also as the script indicates, it doesn’t handle errors that well.

And what do i do about the current logs already in there i have to somehow go through now. Can  it be run on the current logs like this to start it off ?

/path/to/modsec-auditlog-collector.pl /path/to/auditlog/data/  /path/to/auditlog/index

[Ryan Barnett] Just use this command to get all of your past audit log data into the console –

# cat /path/to/apache/logs/index | /path/to/apache/bin/modsec-auditlog-collector.pl

 

This will pipe all of your past audit log data that is held in the INDEX file through the script and it will then submit the logs to the Console.


# This is a proof-of-concept script that listens to the
# audit log in real time and submits the entries to
# a remote HTTP server. This code is not suitable for
# non-trivial production use since it can only submit
# one audit log entry at a time, plus it does not handle
# errors gracefully.
#
# Usage:
#
# 1) Enter the correct parameters $CONSOLE_* below
#
# 2) Configure ModSecurity to use this script for
#    concurrent audit logging index:
#
#    SecAuditLog "|/path/to/modsec-auditlog-collector.pl \
#        /path/to/auditlog/data/ \
#        /path/to/auditlog/index"


Where do i put the info here for a particular sesnor for a particular server if thats how it works, hopefully the data doesnt get jumbled up together ?

my $CONSOLE_URI = "/rpc/auditLogReceiver";
my $CONSOLE_HOST = "192.168.2.11";
my $CONSOLE_PORT = "8886";
my $CONSOLE_USERNAME = "alpha";
my $CONSOLE_PASSWORD = "sensor";


[Ryan Barnett] You need to do the following –

Go into your console and create a new sensor profile.  From the main page, go to Sensors -> then click on the “Add Sensor” button.

Fill out the necessary information for your new sensor.  Important – you must remember the Username and Password that you set for this sensor as you will need this information when setting up the concurrent log forwarding script on your ModSecurity host.

Edit the modsec-auditlog-collector scrip on your modsecurity host  –

    • CONSOLE_HOST needs to have the correct IP address of the host that is running the Console.
    • CONSOLE_USERNAME is the username you specified in the Console when setting up the Sensor profile.
    • CONSOLE_PASSWORD is the password you specified in the Console when setting up the Sensor profile.


Ryan Barnett wrote:

So you are installing the ModSecurity Console on each host that is running ModSecurity?  The idea behind the console is have a central location for remote ModSecurity hosts to send their logs to.  Regardless, the mechanism to use to actually transfer the logs into the console is to use the modsec-auditlog-collector perl script that comes with the ModSecurity 1.9.4 archive.  Take a look at the logging documentation here - http://www.modsecurity.org/documentation/modsecurity-apache/1.9.3/html-multipage/07-logging.html.  Look under the “New Audit Log Type” section for info.

 

--
Ryan C. Barnett
Breach Security: Director of Application Security Training
Web Application Security Consortium (WASC) Member
CIS Apache Benchmark Project Lead
SANS Instructor, GCIA, GCFA, GCIH, GSNA, GCUX, GSEC
Author: Preventing Web Attacks with Apache

 


From: Dan Rossi [mailto:spam@electroteque.org]
Sent: Thursday, December 28, 2006 1:42 AM
To: Ryan Barnett
Cc: mod-security-users@lists.sourceforge.net
Subject: Re: [mod-security-users] how to get console to collect concurrent logs

 

Ryan Barnett wrote:

What do you mean by “collect concurrent logs from a given path”?  Are you referring to how to send concurrent audit log data from ModSecurity hosts to the central Console host?


Hi Ryan, I dont know if you understood it, the console on the localhost of the server does not collect any of the mod security logs this is on all servers i have tried it on. There is definately logs in there though, tonnes of false positives which is why i need this up and running so i can fix it all up.

So basically console runs fine, but cannot load any transactions or any data at all and there is no documentation of what to do next.

I setup some sensor if thats what it needs and selected apache in the pulldown i use apache 2.0.59 and mod sec 2, the interesting thing is in the server-info section it does not display the set configs for mod security could this be the issue , is that how it knows where to get the logs ie i have them being stored on our development machine /var/log/apache2/modsec/console/

etc



 

--
Ryan C. Barnett
Breach Security: Director of Application Security Training
Web Application Security Consortium (WASC) Member
CIS Apache Benchmark Project Lead
SANS Instructor, GCIA, GCFA, GCIH, GSNA, GCUX, GSEC
Author: Preventing Web Attacks with Apache

 


From: mod-security-users-bounces@lists.sourceforge.net [mailto:mod-security-users-bounces@lists.sourceforge.net] On Behalf Of Dan Rossi
Sent: Wednesday, December 27, 2006 7:21 PM
To: mod-security-users@lists.sourceforge.net
Subject: [mod-security-users] how to get console to collect concurrent logs

 

Hi ive asked here quite a few times already, i cant work out how to get the console to collect the concurrent logs from a given path. The console is blank its not collecting and transactions at all, any ideas what do i need to do as there is no log path setting.

Let me know thanks.