Re: [Mod-security-developers] JSON body processor

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Ulisses,

I never had a change to think more about this issue.
Looking for this specific case i really don't think would be a good idea to
create a new logic to ARGS* collections. Not sure what Ryan B. think about
it, but from my point of view, if we need a new logic we must create
specific collections.

ie:  JSON,  JSON_NAMES ....

Is "." (dot)  allowed to create variables names ? I think yes.
If so, we should go json specification and find a better way to create this
logic.  Maybe using "/" ?

ie: user/name, user/manager/name

What do you think ?

Breno

On Sun, Sep 23, 2012 at 12:34 PM, Ulisses Montenegro <
uli...@gm...> wrote:

> Breno & Ryan
>
> Thanks for the pointers. Ryan, I need to look further into how ARGS
> could be used to handle nested data structures. Although deeper
> structures are more common in responses, I've seen some in requests
> too. If we go deeper then 2 levels, then how would we break that data
> into ARGS?
>
> { 'user': {
>     'name': 'John Doe',
>     'email': 'jo...@do...',
>     'manager': {
>         'name': 'Manager John',
>         'email': 'ma...@do...',
>         'company': {
>             'name': 'ModSecurity Corp.',
>             (...)
>         },
>     }
> }
>
> I was thinking that maybe using the fully qualified name for the
> variable might be easier, and would not introduce any artificial
> limitations on the depth on the data structure in the JSON data:
>
> ARGS:user.name = 'John Doe'
> ARGS:user.email = 'jo...@do...'
> ARGS:user.manager.name = 'Manager John'
> ARGS:user.manager.company.name = 'ModSecurity Corp.'
> (...)
>
> Of course, JSON also supports arrays, but since mod_security already
> handles multiple instances of the same parameter, that would not be an
> issue for either option.
>
> Does that make sense, or am I misunderstanding how ARGS work?
>
> Thanks,
> Ulisses
>
> On Sun, Sep 23, 2012 at 1:46 PM, Ryan Barnett <RBa...@tr...>
> wrote:
> > Regarding #2 below - we have two options.
> >
> > 1) A JSON parse could work like the XML parse and access the request body
> > content and simply populate a new collection called JSON.  This is like
> > the XML collection that is simply a long string of text.  The downside of
> > this approach is that here is no context as to what are parameter
> > names/values.  Another option would be to have the JSON parser simply
> > populate this string of text into the current REQUEST_BODY variable.  A
> > rule writer can do this today if they wish using the following example
> > pseudo-rule -
> >
> > SecRule REQUEST_HEADERS:Content-Type "@contains application/json"
> > "phase:1,id:1,nolog,pass,ctl:forceRequestBodyVariable"
> >
> > 2) I think that the best way to do this is to attempt to parse the JSON
> > data into name/value pairs and populate that into ARGS.  If it is parsed
> > in this way, then we don't need to change anything in the current rules.
> >
> > As just one example, I was reviewing the JSON data sent back to twitter
> in
> > response to a Content Security Policy (CSP) violation.  The content-type
> > is application/json and uses the name/value pairs -
> >
> > POST /scribes/csp_report HTTP/1.1
> > Host: twitter.com
> > User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:15.0)
> > Gecko/20100101 Firefox/15.0
> > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> > Accept-Language: en-us,en;q=0.5
> > Accept-Encoding: gzip, deflate
> > DNT: 1
> > Connection: keep-alive
> > Content-Length: 338
> > Content-Type: application/json
> >
> > {"csp-report":{"document-uri":"
> https://mobile.twitter.com/i/templates/m5?re
> > v=1347385509950","referrer":"https://mobile.twitter.com/
> ","blocked-uri":"se
> > lf","violated-directive":"inline
> > <
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
> > %22:%22
> https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
> > ted-directive%22:%22inline> script base
> > restriction","source-file":"
> https://mobile.twitter.com/i/templates/m5?rev=1
> > 347385509950","script-sample":"onclick
> > <
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
> > ample%22:%22onclick> attribute on DIV element"}}
> >
> > Based on this  you would split the name/value pairs by the "Š":"Š."
> > format and have parsed ARGS variable data for use in our rules like -
> >
> > ######################
> > ARGS:csp-report =
> > "document-uri":"
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950
> > ","referrer":"https://mobile.twitter.com/
> ","blocked-uri":"self","violated-d
> > irective":"inline
> > <
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
> > %22:%22
> https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
> > ted-directive%22:%22inline> script base
> > restriction","source-file":"
> https://mobile.twitter.com/i/templates/m5?rev=1
> > 347385509950","script-sample":"onclick
> > <
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
> > ample%22:%22onclick> attribute on DIV element"
> >
> > ARGS:document-uri =
> > https://mobile.twitter.com/i/templates/m5?rev=1347385509950
> > <
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
> > %22:%22
> https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
> > ted-directive%22:%22inline>
> >
> > ARGS:referrer = https://mobile.twitter.com/
> > <
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
> > %22:%22
> https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
> > ted-directive%22:%22inline>
> >
> > ARGS:blocked-uri = self
> >
> > ARGS:violated-directive = inline script base restriction
> >
> > ARGS:source-file =
> > https://mobile.twitter.com/i/templates/m5?rev=1347385509950
> > <
> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
> > ample%22:%22onclick>
> >
> > ARGS:script-sample = onclick attribute on DIV element
> > #######################
> >
> > Hope this helps.
> >
> >
> > --
> > Ryan Barnett
> > Trustwave SpiderLabs
> > ModSecurity Project Leader
> > OWASP ModSecurity CRS Project Leader
> >
> >
> >
> >
> >
> > On 9/23/12 9:31 AM, "Ulisses Montenegro" <uli...@gm...>
> > wrote:
> >
> >>Team
> >>
> >>As my first attempt in contributing to mod_security I've decided to
> >>tackle MODSEC-253, a JSON body processor. I've gone through the XML
> >>and multipart body processors and found them apparently
> >>straightforward. I would like some pointers on issues which I need to
> >>address before deciding on my solution, though.
> >>
> >>1. The XML body processor uses libxml for the actual XML parsing, I
> >>assume adding a JSON parser library would be acceptable as well. If
> >>so, what licenses would be acceptable?
> >>2. XML processor offers a XPath interface for rules to match XML
> >>contents, which is a standard, but AFAIK there is nothing equivalent
> >>for JSON (aside from evaluating Javascript object references). What
> >>interface would work best for the rules to gain access to the JSON
> >>contents?
> >>3. Are there any guidelines/rules regarding memory usage and
> >>performance, i.e., how can if my code or the library I'm using is
> >>performing acceptably? I know I can always benchmark/profile other
> >>body processors and compare the results directly, but I'm looking more
> >>towards hard numbers, if they're available.
> >>4. Finally, do these kind of questions go into JIRA? I decided to try
> >>the mailing list first as I did not want to add possibly irrelevant
> >>information to the JIRA issue, but I think at least items [1] and [2]
> >>should be registered there -- is that how it usually works?
> >>
> >>Thanks a lot for the great work on mod_security
> >>Ulisses
> >>
> >>--
> >>³If debugging is the process of removing software bugs, then
> >>programming must be the process of putting them in.² - Edsger Dijkstra
> >>
>
> >>--------------------------------------------------------------------------
> >>----
> >>Everyone hates slow websites. So do we.
> >>Make your web apps faster with AppDynamics
> >>Download AppDynamics Lite for free today:
> >>http://ad.doubleclick.net/clk;258768047;13503038;j?
> >>http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> >>_______________________________________________
> >>mod-security-developers mailing list
> >>mod...@li...
> >>https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> >>ModSecurity Services from Trustwave's SpiderLabs:
> >>https://www.trustwave.com/spiderLabs.php
> >
> >
> > ________________________________
> >
> > This transmission may contain information that is privileged,
> confidential, and/or exempt from disclosure under applicable law. If you
> are not the intended recipient, you are hereby notified that any
> disclosure, copying, distribution, or use of the information contained
> herein (including any reliance thereon) is STRICTLY PROHIBITED. If you
> received this transmission in error, please immediately contact the sender
> and destroy the material in its entirety, whether in electronic or hard
> copy format.
> >
> >
> >
> ------------------------------------------------------------------------------
> > Everyone hates slow websites. So do we.
> > Make your web apps faster with AppDynamics
> > Download AppDynamics Lite for free today:
> > http://ad.doubleclick.net/clk;258768047;13503038;j?
> > http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> > _______________________________________________
> > mod-security-developers mailing list
> > mod...@li...
> > https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> > ModSecurity Services from Trustwave's SpiderLabs:
> > https://www.trustwave.com/spiderLabs.php
>
>
>
> --
> “If debugging is the process of removing software bugs, then
> programming must be the process of putting them in.” - Edsger Dijkstra
>
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://ad.doubleclick.net/clk;258768047;13503038;j?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> _______________________________________________
> mod-security-developers mailing list
> mod...@li...
> https://lists.sourceforge.net/lists/listinfo/mod-security-developers
> ModSecurity Services from Trustwave's SpiderLabs:
> https://www.trustwave.com/spiderLabs.php
>