Re: [Mod-security-developers] JSON body processor

[Breno S. <bre...@gm...>]

Hello Ulisses,

Did you have any news about this ?
Let me know your ETA to have it done, maybe we can include it into 2.7 code.

Thanks

Breno

On Sun, Sep 23, 2012 at 2:36 PM, Breno Silva <bre...@gm...> wrote:

> Ulisses,
>
> I think something like:
>
> SecRule JSON "@rx test" "..."    <- This will loop and execute operation
> against all JSON variable values
> SecRule JSON:name "@rx test" "..."
> SecRule JSON:user/phone "@rx 123456" "..."
>
> SecRule JSON_RAW "@rx test" "..." -> an unique string with all json data.
> SecRule JSON_NAMES "@rx [a-b] "..."-> collection with variable names
>
> Let see what Ryan think about it from rule creation point of view.
>
> Thanks
>
> Breno
>
>
> On Sun, Sep 23, 2012 at 1:54 PM, Ulisses Montenegro <
> uli...@gm...> wrote:
>
>> Breno,
>>
>> Perhaps it would be easier to look at this the order way around --
>> what would be the most flexible way to write rules for matching JSON
>> data? From a parsing perspective, most libraries offer a
>> JSON-string-to-hashtable approach, which would work for all either
>> scenario.
>>
>> Ryan, do you have any real world use cases for rules matching JSON
>> parameters?
>>
>> Thanks!
>>
>> On Sun, Sep 23, 2012 at 3:38 PM, Breno Silva <bre...@gm...>
>> wrote:
>> > Ulisses,
>> >
>> > I never had a change to think more about this issue.
>> > Looking for this specific case i really don't think would be a good
>> idea to
>> > create a new logic to ARGS* collections. Not sure what Ryan B. think
>> about
>> > it, but from my point of view, if we need a new logic we must create
>> > specific collections.
>> >
>> > ie:  JSON,  JSON_NAMES ....
>> >
>> > Is "." (dot)  allowed to create variables names ? I think yes.
>> > If so, we should go json specification and find a better way to create
>> this
>> > logic.  Maybe using "/" ?
>> >
>> > ie: user/name, user/manager/name
>> >
>> > What do you think ?
>> >
>> > Breno
>> >
>> > On Sun, Sep 23, 2012 at 12:34 PM, Ulisses Montenegro
>> > <uli...@gm...> wrote:
>> >>
>> >> Breno & Ryan
>> >>
>> >> Thanks for the pointers. Ryan, I need to look further into how ARGS
>> >> could be used to handle nested data structures. Although deeper
>> >> structures are more common in responses, I've seen some in requests
>> >> too. If we go deeper then 2 levels, then how would we break that data
>> >> into ARGS?
>> >>
>> >> { 'user': {
>> >>     'name': 'John Doe',
>> >>     'email': 'jo...@do...',
>> >>     'manager': {
>> >>         'name': 'Manager John',
>> >>         'email': 'ma...@do...',
>> >>         'company': {
>> >>             'name': 'ModSecurity Corp.',
>> >>             (...)
>> >>         },
>> >>     }
>> >> }
>> >>
>> >> I was thinking that maybe using the fully qualified name for the
>> >> variable might be easier, and would not introduce any artificial
>> >> limitations on the depth on the data structure in the JSON data:
>> >>
>> >> ARGS:user.name = 'John Doe'
>> >> ARGS:user.email = 'jo...@do...'
>> >> ARGS:user.manager.name = 'Manager John'
>> >> ARGS:user.manager.company.name = 'ModSecurity Corp.'
>> >> (...)
>> >>
>> >> Of course, JSON also supports arrays, but since mod_security already
>> >> handles multiple instances of the same parameter, that would not be an
>> >> issue for either option.
>> >>
>> >> Does that make sense, or am I misunderstanding how ARGS work?
>> >>
>> >> Thanks,
>> >> Ulisses
>> >>
>> >> On Sun, Sep 23, 2012 at 1:46 PM, Ryan Barnett <RBa...@tr...>
>> >> wrote:
>> >> > Regarding #2 below - we have two options.
>> >> >
>> >> > 1) A JSON parse could work like the XML parse and access the request
>> >> > body
>> >> > content and simply populate a new collection called JSON.  This is
>> like
>> >> > the XML collection that is simply a long string of text.  The
>> downside
>> >> > of
>> >> > this approach is that here is no context as to what are parameter
>> >> > names/values.  Another option would be to have the JSON parser simply
>> >> > populate this string of text into the current REQUEST_BODY variable.
>>  A
>> >> > rule writer can do this today if they wish using the following
>> example
>> >> > pseudo-rule -
>> >> >
>> >> > SecRule REQUEST_HEADERS:Content-Type "@contains application/json"
>> >> > "phase:1,id:1,nolog,pass,ctl:forceRequestBodyVariable"
>> >> >
>> >> > 2) I think that the best way to do this is to attempt to parse the
>> JSON
>> >> > data into name/value pairs and populate that into ARGS.  If it is
>> parsed
>> >> > in this way, then we don't need to change anything in the current
>> rules.
>> >> >
>> >> > As just one example, I was reviewing the JSON data sent back to
>> twitter
>> >> > in
>> >> > response to a Content Security Policy (CSP) violation.  The
>> content-type
>> >> > is application/json and uses the name/value pairs -
>> >> >
>> >> > POST /scribes/csp_report HTTP/1.1
>> >> > Host: twitter.com
>> >> > User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:15.0)
>> >> > Gecko/20100101 Firefox/15.0
>> >> > Accept:
>> text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
>> >> > Accept-Language: en-us,en;q=0.5
>> >> > Accept-Encoding: gzip, deflate
>> >> > DNT: 1
>> >> > Connection: keep-alive
>> >> > Content-Length: 338
>> >> > Content-Type: application/json
>> >> >
>> >> >
>> >> > {"csp-report":{"document-uri":"
>> https://mobile.twitter.com/i/templates/m5?re
>> >> >
>> >> > v=1347385509950","referrer":"https://mobile.twitter.com/
>> ","blocked-uri":"se
>> >> > lf","violated-directive":"inline
>> >> >
>> >> > <
>> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
>> >> >
>> >> > %22:%22
>> https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
>> >> > ted-directive%22:%22inline> script base
>> >> >
>> >> > restriction","source-file":"
>> https://mobile.twitter.com/i/templates/m5?rev=1
>> >> > 347385509950","script-sample":"onclick
>> >> >
>> >> > <
>> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
>> >> > ample%22:%22onclick> attribute on DIV element"}}
>> >> >
>> >> > Based on this ­ you would split the name/value pairs by the "Š":"Š."
>> >> > format and have parsed ARGS variable data for use in our rules like -
>> >> >
>> >> > ######################
>> >> > ARGS:csp-report =
>> >> >
>> >> > "document-uri":"
>> https://mobile.twitter.com/i/templates/m5?rev=1347385509950
>> >> >
>> >> > ","referrer":"https://mobile.twitter.com/
>> ","blocked-uri":"self","violated-d
>> >> > irective":"inline
>> >> >
>> >> > <
>> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
>> >> >
>> >> > %22:%22
>> https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
>> >> > ted-directive%22:%22inline> script base
>> >> >
>> >> > restriction","source-file":"
>> https://mobile.twitter.com/i/templates/m5?rev=1
>> >> > 347385509950","script-sample":"onclick
>> >> >
>> >> > <
>> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
>> >> > ample%22:%22onclick> attribute on DIV element"
>> >> >
>> >> > ARGS:document-uri =
>> >> > https://mobile.twitter.com/i/templates/m5?rev=1347385509950
>> >> >
>> >> > <
>> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
>> >> >
>> >> > %22:%22
>> https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
>> >> > ted-directive%22:%22inline>
>> >> >
>> >> > ARGS:referrer = https://mobile.twitter.com/
>> >> >
>> >> > <
>> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22referrer
>> >> >
>> >> > %22:%22
>> https://mobile.twitter.com/%22,%22blocked-uri%22:%22self%22,%22viola
>> >> > ted-directive%22:%22inline>
>> >> >
>> >> > ARGS:blocked-uri = self
>> >> >
>> >> > ARGS:violated-directive = inline script base restriction
>> >> >
>> >> > ARGS:source-file =
>> >> > https://mobile.twitter.com/i/templates/m5?rev=1347385509950
>> >> >
>> >> > <
>> https://mobile.twitter.com/i/templates/m5?rev=1347385509950%22,%22script-s
>> >> > ample%22:%22onclick>
>> >> >
>> >> > ARGS:script-sample = onclick attribute on DIV element
>> >> > #######################
>> >> >
>> >> > Hope this helps.
>> >> >
>> >> >
>> >> > --
>> >> > Ryan Barnett
>> >> > Trustwave SpiderLabs
>> >> > ModSecurity Project Leader
>> >> > OWASP ModSecurity CRS Project Leader
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On 9/23/12 9:31 AM, "Ulisses Montenegro" <
>> uli...@gm...>
>> >> > wrote:
>> >> >
>> >> >>Team
>> >> >>
>> >> >>As my first attempt in contributing to mod_security I've decided to
>> >> >>tackle MODSEC-253, a JSON body processor. I've gone through the XML
>> >> >>and multipart body processors and found them apparently
>> >> >>straightforward. I would like some pointers on issues which I need to
>> >> >>address before deciding on my solution, though.
>> >> >>
>> >> >>1. The XML body processor uses libxml for the actual XML parsing, I
>> >> >>assume adding a JSON parser library would be acceptable as well. If
>> >> >>so, what licenses would be acceptable?
>> >> >>2. XML processor offers a XPath interface for rules to match XML
>> >> >>contents, which is a standard, but AFAIK there is nothing equivalent
>> >> >>for JSON (aside from evaluating Javascript object references). What
>> >> >>interface would work best for the rules to gain access to the JSON
>> >> >>contents?
>> >> >>3. Are there any guidelines/rules regarding memory usage and
>> >> >>performance, i.e., how can if my code or the library I'm using is
>> >> >>performing acceptably? I know I can always benchmark/profile other
>> >> >>body processors and compare the results directly, but I'm looking
>> more
>> >> >>towards hard numbers, if they're available.
>> >> >>4. Finally, do these kind of questions go into JIRA? I decided to try
>> >> >>the mailing list first as I did not want to add possibly irrelevant
>> >> >>information to the JIRA issue, but I think at least items [1] and [2]
>> >> >>should be registered there -- is that how it usually works?
>> >> >>
>> >> >>Thanks a lot for the great work on mod_security
>> >> >>Ulisses
>> >> >>
>> >> >>--
>> >> >>³If debugging is the process of removing software bugs, then
>> >> >>programming must be the process of putting them in.² - Edsger
>> Dijkstra
>> >> >>
>> >>
>> >> >>
>> >>--------------------------------------------------------------------------
>> >> >>----
>> >> >>Everyone hates slow websites. So do we.
>> >> >>Make your web apps faster with AppDynamics
>> >> >>Download AppDynamics Lite for free today:
>> >> >>http://ad.doubleclick.net/clk;258768047;13503038;j?
>> >> >>http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>> >> >>_______________________________________________
>> >> >>mod-security-developers mailing list
>> >> >>mod...@li...
>> >> >>https://lists.sourceforge.net/lists/listinfo/mod-security-developers
>> >> >>ModSecurity Services from Trustwave's SpiderLabs:
>> >> >>https://www.trustwave.com/spiderLabs.php
>> >> >
>> >> >
>> >> > ________________________________
>> >> >
>> >> > This transmission may contain information that is privileged,
>> >> > confidential, and/or exempt from disclosure under applicable law. If
>> you are
>> >> > not the intended recipient, you are hereby notified that any
>> disclosure,
>> >> > copying, distribution, or use of the information contained herein
>> (including
>> >> > any reliance thereon) is STRICTLY PROHIBITED. If you received this
>> >> > transmission in error, please immediately contact the sender and
>> destroy the
>> >> > material in its entirety, whether in electronic or hard copy format.
>> >> >
>> >> >
>> >> >
>> >> >
>> ------------------------------------------------------------------------------
>> >> > Everyone hates slow websites. So do we.
>> >> > Make your web apps faster with AppDynamics
>> >> > Download AppDynamics Lite for free today:
>> >> > http://ad.doubleclick.net/clk;258768047;13503038;j?
>> >> > http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>> >> > _______________________________________________
>> >> > mod-security-developers mailing list
>> >> > mod...@li...
>> >> > https://lists.sourceforge.net/lists/listinfo/mod-security-developers
>> >> > ModSecurity Services from Trustwave's SpiderLabs:
>> >> > https://www.trustwave.com/spiderLabs.php
>> >>
>> >>
>> >>
>> >> --
>> >> “If debugging is the process of removing software bugs, then
>> >> programming must be the process of putting them in.” - Edsger Dijkstra
>> >>
>> >>
>> >>
>> ------------------------------------------------------------------------------
>> >> Everyone hates slow websites. So do we.
>> >> Make your web apps faster with AppDynamics
>> >> Download AppDynamics Lite for free today:
>> >> http://ad.doubleclick.net/clk;258768047;13503038;j?
>> >> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>> >> _______________________________________________
>> >> mod-security-developers mailing list
>> >> mod...@li...
>> >> https://lists.sourceforge.net/lists/listinfo/mod-security-developers
>> >> ModSecurity Services from Trustwave's SpiderLabs:
>> >> https://www.trustwave.com/spiderLabs.php
>> >
>> >
>> >
>> >
>> ------------------------------------------------------------------------------
>> > Everyone hates slow websites. So do we.
>> > Make your web apps faster with AppDynamics
>> > Download AppDynamics Lite for free today:
>> > http://ad.doubleclick.net/clk;258768047;13503038;j?
>> > http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>> > _______________________________________________
>> > mod-security-developers mailing list
>> > mod...@li...
>> > https://lists.sourceforge.net/lists/listinfo/mod-security-developers
>> > ModSecurity Services from Trustwave's SpiderLabs:
>> > https://www.trustwave.com/spiderLabs.php
>>
>>
>>
>> --
>> “If debugging is the process of removing software bugs, then
>> programming must be the process of putting them in.” - Edsger Dijkstra
>>
>>
>> ------------------------------------------------------------------------------
>> Everyone hates slow websites. So do we.
>> Make your web apps faster with AppDynamics
>> Download AppDynamics Lite for free today:
>> http://ad.doubleclick.net/clk;258768047;13503038;j?
>> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
>> _______________________________________________
>> mod-security-developers mailing list
>> mod...@li...
>> https://lists.sourceforge.net/lists/listinfo/mod-security-developers
>> ModSecurity Services from Trustwave's SpiderLabs:
>> https://www.trustwave.com/spiderLabs.php
>>
>
>