Re: [Mod-security-developers] JSON body processor

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Breno,

Perhaps it would be easier to look at this the order way around --
what would be the most flexible way to write rules for matching JSON
data? From a parsing perspective, most libraries offer a
JSON-string-to-hashtable approach, which would work for all either
scenario.

Ryan, do you have any real world use cases for rules matching JSON parameters?

Thanks!

On Sun, Sep 23, 2012 at 3:38 PM, Breno Silva <bre...@gm...> wrote:
> Ulisses,
>
> I never had a change to think more about this issue.
> Looking for this specific case i really don't think would be a good idea to
> create a new logic to ARGS* collections. Not sure what Ryan B. think about
> it, but from my point of view, if we need a new logic we must create
> specific collections.
>
> ie:  JSON,  JSON_NAMES ....
>
> Is "." (dot)  allowed to create variables names ? I think yes.
> If so, we should go json specification and find a better way to create this
> logic.  Maybe using "/" ?
>
> ie: user/name, user/manager/name
>
> What do you think ?
>
> Breno
>
> On Sun, Sep 23, 2012 at 12:34 PM, Ulisses Montenegro
> <uli...@gm...> wrote:
>>
>> Breno & Ryan
>>
>> Thanks for the pointers. Ryan, I need to look further into how ARGS
>> could be used to handle nested data structures. Although deeper
>> structures are more common in responses, I've seen some in requests
>> too. If we go deeper then 2 levels, then how would we break that data
>> into ARGS?
>>
>> { 'user': {
>>     'name': 'John Doe',
>>     'email': 'jo...@do...',
>>     'manager': {
>>         'name': 'Manager John',
>>         'email': 'ma...@do...',
>>         'company': {
>>             'name': 'ModSecurity Corp.',
>>             (...)
>>         },
>>     }
>> }
>>
>> I was thinking that maybe using the fully qualified name for the
>> variable might be easier, and would not introduce any artificial
>> limitations on the depth on the data structure in the JSON data:
>>
>> ARGS:user.name = 'John Doe'
>> ARGS:user.email = 'jo...@do...'
>> ARGS:user.manager.name = 'Manager John'
>> ARGS:user.manager.company.name = 'ModSecurity Corp.'
>> (...)
>>
>> Of course, JSON also supports arrays, but since mod_security already
>> handles multiple instances of the same parameter, that would not be an
>> issue for either option.
>>
>> Does that make sense, or am I misunderstanding how ARGS work?
>>
>> Thanks,
>> Ulisses
>>
>> On Sun, Sep 23, 2012 at 1:46 PM, Ryan Barnett <RBa...@tr...>
>> wrote:
>> > Regarding #2 below - we have two options.
>> >
>> > 1) A JSON parse could work like the XML parse and access the request
>> > body
>> > content and simply populate a new collection called JSON.  This is like
>> > the XML collection that is simply a long string of text.  The downside
>> > of
>> > this approach is that here is no context as to what are parameter
>> > names/values.  Another option would be to have the JSON parser simply
>> > populate this string of text into the current REQUEST_BODY variable.  A
>> > rule writer can do this today if they wish using the following example
>> > pseudo-rule -
>> >
>> > SecRule REQUEST_HEADERS:Content-Type "@contains application/json"
>> > "phase:1,id:1,nolog,pass,ctl:forceRequestBodyVariable"
>> >
>> > 2) I think that the best way to do this is to attempt to parse the JSON
>> > data into name/value pairs and populate that into ARGS.  If it is parsed
>> > in this way, then we don't need to change anything in the current rules.
>> >
>> > As just one example, I was reviewing the JSON data sent back to twitter
>> > in
>> > response to a Content Security Policy (CSP) violation.  The content-type
>> > is application/json and uses the name/value pairs -
>> >
>> > POST /scribes/csp_report HTTP/1.1
>> > Host: twitter.com
>> > User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:15.0)
>> > Gecko/20100101 Firefox/15.0
>> > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
>> > Accept-Language: en-us,en;q=0.5
>> > Accept-Encoding: gzip, deflate
>> > DNT: 1
>> > Connection: keep-alive
>> > Content-Length: 338
>> > Content-Type: application/json
>> >
>> >
>> > {"csp-report":{"document-uri":"https://mobile.twitter.com/i/templates/m5?re
>> >
>> > v=1347385509950","referrer":"https://mobile.twitter.com/","blocked-uri":"se
>> > lf","violated-directive":"inline
>> >
>> > <https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
>> >
>> > %22:%22https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
>> > ted-directive%22:%22inline> script base
>> >
>> > restriction","source-file":"https://mobile.twitter.com/i/templates/m5?rev=1
>> > 347385509950","script-sample":"onclick
>> >
>> > <https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
>> > ample%22:%22onclick> attribute on DIV element"}}
>> >
>> > Based on this  you would split the name/value pairs by the "Š":"Š."
>> > format and have parsed ARGS variable data for use in our rules like -
>> >
>> > ######################
>> > ARGS:csp-report =
>> >
>> > "document-uri":"https://mobile.twitter.com/i/templates/m5?rev=1347385509950
>> >
>> > ","referrer":"https://mobile.twitter.com/","blocked-uri":"self","violated-d
>> > irective":"inline
>> >
>> > <https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
>> >
>> > %22:%22https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
>> > ted-directive%22:%22inline> script base
>> >
>> > restriction","source-file":"https://mobile.twitter.com/i/templates/m5?rev=1
>> > 347385509950","script-sample":"onclick
>> >
>> > <https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
>> > ample%22:%22onclick> attribute on DIV element"
>> >
>> > ARGS:document-uri =
>> > https://mobile.twitter.com/i/templates/m5?rev=1347385509950
>> >
>> > <https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
>> >
>> > %22:%22https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
>> > ted-directive%22:%22inline>
>> >
>> > ARGS:referrer = https://mobile.twitter.com/
>> >
>> > <https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
>> >
>> > %22:%22https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
>> > ted-directive%22:%22inline>
>> >
>> > ARGS:blocked-uri = self
>> >
>> > ARGS:violated-directive = inline script base restriction
>> >
>> > ARGS:source-file =
>> > https://mobile.twitter.com/i/templates/m5?rev=1347385509950
>> >
>> > <https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
>> > ample%22:%22onclick>
>> >
>> > ARGS:script-sample = onclick attribute on DIV element
>> > #######################
>> >
>> > Hope this helps.
>> >
>> >
>> > --
>> > Ryan Barnett
>> > Trustwave SpiderLabs
>> > ModSecurity Project Leader
>> > OWASP ModSecurity CRS Project Leader
>> >
>> >
>> >
>> >
>> >
>> > On 9/23/12 9:31 AM, "Ulisses Montenegro" <uli...@gm...>
>> > wrote:
>> >
>> >>Team
>> >>
>> >>As my first attempt in contributing to mod_security I've decided to
>> >>tackle MODSEC-253, a JSON body processor. I've gone through the XML
>> >>and multipart body processors and found them apparently
>> >>straightforward. I would like some pointers on issues which I need to
>> >>address before deciding on my solution, though.
>> >>
>> >>1. The XML body processor uses libxml for the actual XML parsing, I
>> >>assume adding a JSON parser library would be acceptable as well. If
>> >>so, what licenses would be acceptable?
>> >>2. XML processor offers a XPath interface for rules to match XML
>> >>contents, which is a standard, but AFAIK there is nothing equivalent
>> >>for JSON (aside from evaluating Javascript object references). What
>> >>interface would work best for the rules to gain access to the JSON
>> >>contents?
>> >>3. Are there any guidelines/rules regarding memory usage and
>> >>performance, i.e., how can if my code or the library I'm using is
>> >>performing acceptably? I know I can always benchmark/profile other
>> >>body processors and compare the results directly, but I'm looking more
>> >>towards hard numbers, if they're available.
>> >>4. Finally, do these kind of questions go into JIRA? I decided to try
>> >>the mailing list first as I did not want to add possibly irrelevant
>> >>information to the JIRA issue, but I think at least items [1] and [2]
>> >>should be registered there -- is that how it usually works?
>> >>
>> >>Thanks a lot for the great work on mod_security
>> >>Ulisses
>> >>
>> >>--
>> >>³If debugging is the process of removing software bugs, then
>> >>programming must be the process of putting them in.² - Edsger Dijkstra
>> >>
>>
>> >> >>--------------------------------------------------------------------------
>> >>----
>> >>Everyone hates slow websites. So do we.
>> >>Make your web apps faster with AppDynamics
>> >>Download AppDynamics Lite for free today:
>> >>http://ad.doubleclick.net/clk;258768047;13503038;j?
>> >>http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>> >>_______________________________________________
>> >>mod-security-developers mailing list
>> >>mod...@li...
>> >>https://lists.sourceforge.net/lists/listinfo/mod-security-developers
>> >>ModSecurity Services from Trustwave's SpiderLabs:
>> >>https://www.trustwave.com/spiderLabs.php
>> >
>> >
>> > ________________________________
>> >
>> > This transmission may contain information that is privileged,
>> > confidential, and/or exempt from disclosure under applicable law. If you are
>> > not the intended recipient, you are hereby notified that any disclosure,
>> > copying, distribution, or use of the information contained herein (including
>> > any reliance thereon) is STRICTLY PROHIBITED. If you received this
>> > transmission in error, please immediately contact the sender and destroy the
>> > material in its entirety, whether in electronic or hard copy format.
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Everyone hates slow websites. So do we.
>> > Make your web apps faster with AppDynamics
>> > Download AppDynamics Lite for free today:
>> > http://ad.doubleclick.net/clk;258768047;13503038;j?
>> > http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>> > _______________________________________________
>> > mod-security-developers mailing list
>> > mod...@li...
>> > https://lists.sourceforge.net/lists/listinfo/mod-security-developers
>> > ModSecurity Services from Trustwave's SpiderLabs:
>> > https://www.trustwave.com/spiderLabs.php
>>
>>
>>
>> --
>> “If debugging is the process of removing software bugs, then
>> programming must be the process of putting them in.” - Edsger Dijkstra
>>
>>
>> ------------------------------------------------------------------------------
>> Everyone hates slow websites. So do we.
>> Make your web apps faster with AppDynamics
>> Download AppDynamics Lite for free today:
>> http://ad.doubleclick.net/clk;258768047;13503038;j?
>> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>> _______________________________________________
>> mod-security-developers mailing list
>> mod...@li...
>> https://lists.sourceforge.net/lists/listinfo/mod-security-developers
>> ModSecurity Services from Trustwave's SpiderLabs:
>> https://www.trustwave.com/spiderLabs.php
>
>
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://ad.doubleclick.net/clk;258768047;13503038;j?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> _______________________________________________
> mod-security-developers mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> ModSecurity Services from Trustwave's SpiderLabs:
> https://www.trustwave.com/spiderLabs.php

-- 
“If debugging is the process of removing software bugs, then
programming must be the process of putting them in.” - Edsger Dijkstra