Re: [Mod-security-developers] JSON body processor

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Ulisses,

I think something like:

SecRule JSON "@rx test" "..."    <- This will loop and execute operation
against all JSON variable values
SecRule JSON:name "@rx test" "..."
SecRule JSON:user/phone "@rx 123456" "..."

SecRule JSON_RAW "@rx test" "..." -> an unique string with all json data.
SecRule JSON_NAMES "@rx [a-b] "..."-> collection with variable names

Let see what Ryan think about it from rule creation point of view.

Thanks

Breno

On Sun, Sep 23, 2012 at 1:54 PM, Ulisses Montenegro <
uli...@gm...> wrote:

> Breno,
>
> Perhaps it would be easier to look at this the order way around --
> what would be the most flexible way to write rules for matching JSON
> data? From a parsing perspective, most libraries offer a
> JSON-string-to-hashtable approach, which would work for all either
> scenario.
>
> Ryan, do you have any real world use cases for rules matching JSON
> parameters?
>
> Thanks!
>
> On Sun, Sep 23, 2012 at 3:38 PM, Breno Silva <bre...@gm...>
> wrote:
> > Ulisses,
> >
> > I never had a change to think more about this issue.
> > Looking for this specific case i really don't think would be a good idea
> to
> > create a new logic to ARGS* collections. Not sure what Ryan B. think
> about
> > it, but from my point of view, if we need a new logic we must create
> > specific collections.
> >
> > ie:  JSON,  JSON_NAMES ....
> >
> > Is "." (dot)  allowed to create variables names ? I think yes.
> > If so, we should go json specification and find a better way to create
> this
> > logic.  Maybe using "/" ?
> >
> > ie: user/name, user/manager/name
> >
> > What do you think ?
> >
> > Breno
> >
> > On Sun, Sep 23, 2012 at 12:34 PM, Ulisses Montenegro
> > <uli...@gm...> wrote:
> >>
> >> Breno & Ryan
> >>
> >> Thanks for the pointers. Ryan, I need to look further into how ARGS
> >> could be used to handle nested data structures. Although deeper
> >> structures are more common in responses, I've seen some in requests
> >> too. If we go deeper then 2 levels, then how would we break that data
> >> into ARGS?
> >>
> >> { 'user': {
> >>     'name': 'John Doe',
> >>     'email': 'jo...@do...',
> >>     'manager': {
> >>         'name': 'Manager John',
> >>         'email': 'ma...@do...',
> >>         'company': {
> >>             'name': 'ModSecurity Corp.',
> >>             (...)
> >>         },
> >>     }
> >> }
> >>
> >> I was thinking that maybe using the fully qualified name for the
> >> variable might be easier, and would not introduce any artificial
> >> limitations on the depth on the data structure in the JSON data:
> >>
> >> ARGS:user.name = 'John Doe'
> >> ARGS:user.email = 'jo...@do...'
> >> ARGS:user.manager.name = 'Manager John'
> >> ARGS:user.manager.company.name = 'ModSecurity Corp.'
> >> (...)
> >>
> >> Of course, JSON also supports arrays, but since mod_security already
> >> handles multiple instances of the same parameter, that would not be an
> >> issue for either option.
> >>
> >> Does that make sense, or am I misunderstanding how ARGS work?
> >>
> >> Thanks,
> >> Ulisses
> >>
> >> On Sun, Sep 23, 2012 at 1:46 PM, Ryan Barnett <RBa...@tr...>
> >> wrote:
> >> > Regarding #2 below - we have two options.
> >> >
> >> > 1) A JSON parse could work like the XML parse and access the request
> >> > body
> >> > content and simply populate a new collection called JSON.  This is
> like
> >> > the XML collection that is simply a long string of text.  The downside
> >> > of
> >> > this approach is that here is no context as to what are parameter
> >> > names/values.  Another option would be to have the JSON parser simply
> >> > populate this string of text into the current REQUEST_BODY variable.
>  A
> >> > rule writer can do this today if they wish using the following example
> >> > pseudo-rule -
> >> >
> >> > SecRule REQUEST_HEADERS:Content-Type "@contains application/json"
> >> > "phase:1,id:1,nolog,pass,ctl:forceRequestBodyVariable"
> >> >
> >> > 2) I think that the best way to do this is to attempt to parse the
> JSON
> >> > data into name/value pairs and populate that into ARGS.  If it is
> parsed
> >> > in this way, then we don't need to change anything in the current
> rules.
> >> >
> >> > As just one example, I was reviewing the JSON data sent back to
> twitter
> >> > in
> >> > response to a Content Security Policy (CSP) violation.  The
> content-type
> >> > is application/json and uses the name/value pairs -
> >> >
> >> > POST /scribes/csp_report HTTP/1.1
> >> > Host: twitter.com
> >> > User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:15.0)
> >> > Gecko/20100101 Firefox/15.0
> >> > Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> >> > Accept-Language: en-us,en;q=0.5
> >> > Accept-Encoding: gzip, deflate
> >> > DNT: 1
> >> > Connection: keep-alive
> >> > Content-Length: 338
> >> > Content-Type: application/json
> >> >
> >> >
> >> > {"csp-report":{"document-uri":"
> https://mobile.twitter.com/i/templates/m5?re
> >> >
> >> > v=1347385509950","referrer":"https://mobile.twitter.com/
> ","blocked-uri":"se
> >> > lf","violated-directive":"inline
> >> >
> >> > <
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
> >> >
> >> > %22:%22
> https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
> >> > ted-directive%22:%22inline> script base
> >> >
> >> > restriction","source-file":"
> https://mobile.twitter.com/i/templates/m5?rev=1
> >> > 347385509950","script-sample":"onclick
> >> >
> >> > <
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
> >> > ample%22:%22onclick> attribute on DIV element"}}
> >> >
> >> > Based on this  you would split the name/value pairs by the "Š":"Š."
> >> > format and have parsed ARGS variable data for use in our rules like -
> >> >
> >> > ######################
> >> > ARGS:csp-report =
> >> >
> >> > "document-uri":"
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950
> >> >
> >> > ","referrer":"https://mobile.twitter.com/
> ","blocked-uri":"self","violated-d
> >> > irective":"inline
> >> >
> >> > <
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
> >> >
> >> > %22:%22
> https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
> >> > ted-directive%22:%22inline> script base
> >> >
> >> > restriction","source-file":"
> https://mobile.twitter.com/i/templates/m5?rev=1
> >> > 347385509950","script-sample":"onclick
> >> >
> >> > <
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
> >> > ample%22:%22onclick> attribute on DIV element"
> >> >
> >> > ARGS:document-uri =
> >> > https://mobile.twitter.com/i/templates/m5?rev=1347385509950
> >> >
> >> > <
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
> >> >
> >> > %22:%22
> https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
> >> > ted-directive%22:%22inline>
> >> >
> >> > ARGS:referrer = https://mobile.twitter.com/
> >> >
> >> > <
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
> >> >
> >> > %22:%22
> https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
> >> > ted-directive%22:%22inline>
> >> >
> >> > ARGS:blocked-uri = self
> >> >
> >> > ARGS:violated-directive = inline script base restriction
> >> >
> >> > ARGS:source-file =
> >> > https://mobile.twitter.com/i/templates/m5?rev=1347385509950
> >> >
> >> > <
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
> >> > ample%22:%22onclick>
> >> >
> >> > ARGS:script-sample = onclick attribute on DIV element
> >> > #######################
> >> >
> >> > Hope this helps.
> >> >
> >> >
> >> > --
> >> > Ryan Barnett
> >> > Trustwave SpiderLabs
> >> > ModSecurity Project Leader
> >> > OWASP ModSecurity CRS Project Leader
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On 9/23/12 9:31 AM, "Ulisses Montenegro" <
> uli...@gm...>
> >> > wrote:
> >> >
> >> >>Team
> >> >>
> >> >>As my first attempt in contributing to mod_security I've decided to
> >> >>tackle MODSEC-253, a JSON body processor. I've gone through the XML
> >> >>and multipart body processors and found them apparently
> >> >>straightforward. I would like some pointers on issues which I need to
> >> >>address before deciding on my solution, though.
> >> >>
> >> >>1. The XML body processor uses libxml for the actual XML parsing, I
> >> >>assume adding a JSON parser library would be acceptable as well. If
> >> >>so, what licenses would be acceptable?
> >> >>2. XML processor offers a XPath interface for rules to match XML
> >> >>contents, which is a standard, but AFAIK there is nothing equivalent
> >> >>for JSON (aside from evaluating Javascript object references). What
> >> >>interface would work best for the rules to gain access to the JSON
> >> >>contents?
> >> >>3. Are there any guidelines/rules regarding memory usage and
> >> >>performance, i.e., how can if my code or the library I'm using is
> >> >>performing acceptably? I know I can always benchmark/profile other
> >> >>body processors and compare the results directly, but I'm looking more
> >> >>towards hard numbers, if they're available.
> >> >>4. Finally, do these kind of questions go into JIRA? I decided to try
> >> >>the mailing list first as I did not want to add possibly irrelevant
> >> >>information to the JIRA issue, but I think at least items [1] and [2]
> >> >>should be registered there -- is that how it usually works?
> >> >>
> >> >>Thanks a lot for the great work on mod_security
> >> >>Ulisses
> >> >>
> >> >>--
> >> >>³If debugging is the process of removing software bugs, then
> >> >>programming must be the process of putting them in.² - Edsger Dijkstra
> >> >>
> >>
> >> >>
> >>--------------------------------------------------------------------------
> >> >>----
> >> >>Everyone hates slow websites. So do we.
> >> >>Make your web apps faster with AppDynamics
> >> >>Download AppDynamics Lite for free today:
> >> >>http://ad.doubleclick.net/clk;258768047;13503038;j?
> >> >>http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> >> >>_______________________________________________
> >> >>mod-security-developers mailing list
> >> >>mod...@li...
> >> >>https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> >> >>ModSecurity Services from Trustwave's SpiderLabs:
> >> >>https://www.trustwave.com/spiderLabs.php
> >> >
> >> >
> >> > ________________________________
> >> >
> >> > This transmission may contain information that is privileged,
> >> > confidential, and/or exempt from disclosure under applicable law. If
> you are
> >> > not the intended recipient, you are hereby notified that any
> disclosure,
> >> > copying, distribution, or use of the information contained herein
> (including
> >> > any reliance thereon) is STRICTLY PROHIBITED. If you received this
> >> > transmission in error, please immediately contact the sender and
> destroy the
> >> > material in its entirety, whether in electronic or hard copy format.
> >> >
> >> >
> >> >
> >> >
> ------------------------------------------------------------------------------
> >> > Everyone hates slow websites. So do we.
> >> > Make your web apps faster with AppDynamics
> >> > Download AppDynamics Lite for free today:
> >> > http://ad.doubleclick.net/clk;258768047;13503038;j?
> >> > http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> >> > _______________________________________________
> >> > mod-security-developers mailing list
> >> > mod...@li...
> >> > https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> >> > ModSecurity Services from Trustwave's SpiderLabs:
> >> > https://www.trustwave.com/spiderLabs.php
> >>
> >>
> >>
> >> --
> >> “If debugging is the process of removing software bugs, then
> >> programming must be the process of putting them in.” - Edsger Dijkstra
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Everyone hates slow websites. So do we.
> >> Make your web apps faster with AppDynamics
> >> Download AppDynamics Lite for free today:
> >> http://ad.doubleclick.net/clk;258768047;13503038;j?
> >> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> >> _______________________________________________
> >> mod-security-developers mailing list
> >> mod...@li...
> >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> >> ModSecurity Services from Trustwave's SpiderLabs:
> >> https://www.trustwave.com/spiderLabs.php
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > Everyone hates slow websites. So do we.
> > Make your web apps faster with AppDynamics
> > Download AppDynamics Lite for free today:
> > http://ad.doubleclick.net/clk;258768047;13503038;j?
> > http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> > _______________________________________________
> > mod-security-developers mailing list
> > mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> > ModSecurity Services from Trustwave's SpiderLabs:
> > https://www.trustwave.com/spiderLabs.php
>
>
>
> --
> “If debugging is the process of removing software bugs, then
> programming must be the process of putting them in.” - Edsger Dijkstra
>
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://ad.doubleclick.net/clk;258768047;13503038;j?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> _______________________________________________
> mod-security-developers mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> ModSecurity Services from Trustwave's SpiderLabs:
> https://www.trustwave.com/spiderLabs.php
>