Re: [Mod-security-developers] JSON body processor

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Breno & Ryan

Thanks for the pointers. Ryan, I need to look further into how ARGS
could be used to handle nested data structures. Although deeper
structures are more common in responses, I've seen some in requests
too. If we go deeper then 2 levels, then how would we break that data
into ARGS?

{ 'user': {
    'name': 'John Doe',
    'email': 'jo...@do...',
    'manager': {
        'name': 'Manager John',
        'email': 'ma...@do...',
        'company': {
            'name': 'ModSecurity Corp.',
            (...)
        },
    }
}

I was thinking that maybe using the fully qualified name for the
variable might be easier, and would not introduce any artificial
limitations on the depth on the data structure in the JSON data:

ARGS:user.name = 'John Doe'
ARGS:user.email = 'jo...@do...'
ARGS:user.manager.name = 'Manager John'
ARGS:user.manager.company.name = 'ModSecurity Corp.'
(...)

Of course, JSON also supports arrays, but since mod_security already
handles multiple instances of the same parameter, that would not be an
issue for either option.

Does that make sense, or am I misunderstanding how ARGS work?

Thanks,
Ulisses

On Sun, Sep 23, 2012 at 1:46 PM, Ryan Barnett <RBa...@tr...> wrote:
> Regarding #2 below - we have two options.
>
> 1) A JSON parse could work like the XML parse and access the request body
> content and simply populate a new collection called JSON.  This is like
> the XML collection that is simply a long string of text.  The downside of
> this approach is that here is no context as to what are parameter
> names/values.  Another option would be to have the JSON parser simply
> populate this string of text into the current REQUEST_BODY variable.  A
> rule writer can do this today if they wish using the following example
> pseudo-rule -
>
> SecRule REQUEST_HEADERS:Content-Type "@contains application/json"
> "phase:1,id:1,nolog,pass,ctl:forceRequestBodyVariable"
>
> 2) I think that the best way to do this is to attempt to parse the JSON
> data into name/value pairs and populate that into ARGS.  If it is parsed
> in this way, then we don't need to change anything in the current rules.
>
> As just one example, I was reviewing the JSON data sent back to twitter in
> response to a Content Security Policy (CSP) violation.  The content-type
> is application/json and uses the name/value pairs -
>
> POST /scribes/csp_report HTTP/1.1
> Host: twitter.com
> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:15.0)
> Gecko/20100101 Firefox/15.0
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> Accept-Language: en-us,en;q=0.5
> Accept-Encoding: gzip, deflate
> DNT: 1
> Connection: keep-alive
> Content-Length: 338
> Content-Type: application/json
>
> {"csp-report":{"document-uri":"https://mobile.twitter.com/i/templates/m5?re
> v=1347385509950","referrer":"https://mobile.twitter.com/","blocked-uri":"se
> lf","violated-directive":"inline
> <https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
> %22:%22https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
> ted-directive%22:%22inline> script base
> restriction","source-file":"https://mobile.twitter.com/i/templates/m5?rev=1
> 347385509950","script-sample":"onclick
> <https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
> ample%22:%22onclick> attribute on DIV element"}}
>
> Based on this  you would split the name/value pairs by the "Š":"Š."
> format and have parsed ARGS variable data for use in our rules like -
>
> ######################
> ARGS:csp-report =
> "document-uri":"https://mobile.twitter.com/i/templates/m5?rev=1347385509950
> ","referrer":"https://mobile.twitter.com/","blocked-uri":"self","violated-d
> irective":"inline
> <https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
> %22:%22https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
> ted-directive%22:%22inline> script base
> restriction","source-file":"https://mobile.twitter.com/i/templates/m5?rev=1
> 347385509950","script-sample":"onclick
> <https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
> ample%22:%22onclick> attribute on DIV element"
>
> ARGS:document-uri =
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950
> <https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
> %22:%22https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
> ted-directive%22:%22inline>
>
> ARGS:referrer = https://mobile.twitter.com/
> <https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
> %22:%22https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
> ted-directive%22:%22inline>
>
> ARGS:blocked-uri = self
>
> ARGS:violated-directive = inline script base restriction
>
> ARGS:source-file =
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950
> <https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
> ample%22:%22onclick>
>
> ARGS:script-sample = onclick attribute on DIV element
> #######################
>
> Hope this helps.
>
>
> --
> Ryan Barnett
> Trustwave SpiderLabs
> ModSecurity Project Leader
> OWASP ModSecurity CRS Project Leader
>
>
>
>
>
> On 9/23/12 9:31 AM, "Ulisses Montenegro" <uli...@gm...>
> wrote:
>
>>Team
>>
>>As my first attempt in contributing to mod_security I've decided to
>>tackle MODSEC-253, a JSON body processor. I've gone through the XML
>>and multipart body processors and found them apparently
>>straightforward. I would like some pointers on issues which I need to
>>address before deciding on my solution, though.
>>
>>1. The XML body processor uses libxml for the actual XML parsing, I
>>assume adding a JSON parser library would be acceptable as well. If
>>so, what licenses would be acceptable?
>>2. XML processor offers a XPath interface for rules to match XML
>>contents, which is a standard, but AFAIK there is nothing equivalent
>>for JSON (aside from evaluating Javascript object references). What
>>interface would work best for the rules to gain access to the JSON
>>contents?
>>3. Are there any guidelines/rules regarding memory usage and
>>performance, i.e., how can if my code or the library I'm using is
>>performing acceptably? I know I can always benchmark/profile other
>>body processors and compare the results directly, but I'm looking more
>>towards hard numbers, if they're available.
>>4. Finally, do these kind of questions go into JIRA? I decided to try
>>the mailing list first as I did not want to add possibly irrelevant
>>information to the JIRA issue, but I think at least items [1] and [2]
>>should be registered there -- is that how it usually works?
>>
>>Thanks a lot for the great work on mod_security
>>Ulisses
>>
>>--
>>³If debugging is the process of removing software bugs, then
>>programming must be the process of putting them in.² - Edsger Dijkstra
>>
>>--------------------------------------------------------------------------
>>----
>>Everyone hates slow websites. So do we.
>>Make your web apps faster with AppDynamics
>>Download AppDynamics Lite for free today:
>>http://ad.doubleclick.net/clk;258768047;13503038;j?
>>http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>>_______________________________________________
>>mod-security-developers mailing list
>>mod...@li...
>>https://lists.sourceforge.net/lists/listinfo/mod-security-developers
>>ModSecurity Services from Trustwave's SpiderLabs:
>>https://www.trustwave.com/spiderLabs.php
>
>
> ________________________________
>
> This transmission may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format.
>
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://ad.doubleclick.net/clk;258768047;13503038;j?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> _______________________________________________
> mod-security-developers mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> ModSecurity Services from Trustwave's SpiderLabs:
> https://www.trustwave.com/spiderLabs.php

-- 
“If debugging is the process of removing software bugs, then
programming must be the process of putting them in.” - Edsger Dijkstra