Re: [Mod-security-developers] Implementing IDMEF

[Felipe C. <FC...@tr...>]

Hi Vérène,

Thank you for your interest, contributions are very welcomed.

In ModSecurity we have different logs:

1 ­ Web server log: 1-line log which contains very brief information
(apache error log).
2 ­ Audit logs: Depending on the configuration, it can save an entire
transaction [8].
3 ­ Debug logs: Recommended to be used while creating/editing/debugging
rules.


It seems that you are interested in the audit logs. We have some issues
opened
on our github regarding saving the audit logs in formats that are not its
original
one, Including JSON and XML.

Since there are different needs in terms of log format, we start to
implement an
Utility named mlgoc-ng [6]. This utility works with pipelines where the
first
element is a producer (responsible for reading and parser the logs), and
the
subsequent elements can process the log information (including transform
it into
another formats, such as: JSON). In that pipeline the last element can
save the
content on disk and/or delivery the content to a central log server.

Notice that there are two different things: mlgoc [7] and mlogc-ng. Mlogc
is an
utility that basically allows you to send the logs over the network. This
is
what is used to send logs to AuditConsole [1] and WAF-FLE [2]. (There are
alternatives, as example of: [3]).


Other interesting approach is the utilization of the logstash [4]. There
is this
logstash-modsecurity [5] which is capable to understand the format on
audit logs
and transform it in something else.



I believe that the best option is the utilization of the mlogc-ng as it
already
reads the log for you and gives you an c-structure. Apparently you only
have to
code a Mlogc-ng element that will disposal the data in the XML format
(expressed
on the RFC 4765). Optionally you can create a second module that will be
able to
send this xml to a end server.


The mlogc-ng pipeline will be something like:

read_from_filesystem -> transform to RFC 4765 format -> dispatch to an end
server


Alternatively:

read_from_filesystem -> transform to RFC 4765 format -> save to disk


Here is an example of how an mlogc-ng element looks like:
https://github.com/SpiderLabs/modsecurity-mlogc-ng/blob/master/pipe_element
s/send_to_server.c



Let me know If you need any help to implement this, I will be glad to help.


[1] https://jwall.org/web/audit/console/index.jsp
[2] http://waf-fle.org/
[3] https://jwall.org/tools/jwall-tools.jsp
[4] http://logstash.net/
[5] https://github.com/bitsofinfo/logstash-modsecurity
[6] https://github.com/SpiderLabs/modsecurity-mlogc-ng
[7] https://github.com/SpiderLabs/ModSecurity/tree/master/mlogc
[8]
https://github.com/SpiderLabs/ModSecurity/wiki/ModSecurity-2-Data-Formats#A
udit_Log


Thanks,
Felipe ³Zimmerle² Costa
Security Researcher, SpiderLabs

Trustwave | SMART SECURITY ON DEMAND
www.trustwave.com <http://www.trustwave.com/>





From:  Vérène Houdebine <ver...@re...>
Reply-To:  "mod...@li..."
<mod...@li...>
Date:  Wednesday, March 4, 2015 at 1:52 PM
To:  "mod...@li..."
<mod...@li...>
Subject:  [Mod-security-developers] Implementing IDMEF


Hello,



We are interested in implementing the IDMEF format in modsecurity.
For those of you who don¹t know it yet, IDMEF is a data format defined in
the RFC 4765.
That would mean adding an option to avoid regular log and, instead,
directly send IDMEF alerts to a prelude manager.
We are working on this project in the context of our student¹s project.

Is it adequate to add a new module? Do you have any idea/advice/suggestion
as to where we can fetch the information? Or do you have some
complementary documentation about how the code is
 organized?



Thank you very much in advance!
Vérène


________________________________

This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is strictly prohibited. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format.