Re: [mod-security-users] web app discovery

[kiran k <kir...@ya...>]

 Thanks Ryan, I tried htmlspider on mail.yahoo.com, it didn't generate any rules  (form action points to https URL though). I tried on other plain website, which has login.php (no https, but it had uid, passwd parameters) that also didn't lead to any rules.
   
  I will get your book, will be worth every penny. Any ideas what is missing ?
  
 
  
 

Ryan Barnett <rcb...@gm...> wrote: <Shameless Plug> 
 I outline both manual and automated ruleset creations in my book "Preventing Web Attacks with Apache" -
 http://www.amazon.com/gp/product/0321321286/102-5050782-8736967?v=glance&n=283155
 </Shameless Plug>
  
 You can use the Mod_Parmguard htmlspider perl script to do this -
 http://www.trickytools.com/parmguard/manual-1.3/generator.html
  
 This will get you most of the way there as it will crawl the site recursively, extract out all of the input forms, etc... and create whitelisted XML based rulesets for Mod_Parmguard to use.  You will then have to translate these into the comparable modsecurity format. 
  
 For example -
  
  
  # ./htmlspider.pl -h http://192.168.1.102/cgi-bin/wm.cgi 

  <?xml version="1.0"?>

  <!DOCTYPE parmguard SYSTEM "mod_parmguard.dtd">

   

  <!-- =============================================================== -->

  <!-- SCANNING SUMMARY                                                -->

   <!-- mod_parmguard Generator, version 1.2                            -->

   <!-- Date of Scan: Thu May 12 15:54:52 2005                          -->

   <!-- Start URL: http://192.168.1.102/cgi-bin/wm.cgi                  -->

  <!-- List of not parsed URLs                                         -->

   <!-- =============================================================== -->

   

  <parmguard>

          <url>

                  <match>^/cgi-bin/wm.cgi</match>

                  <parm name=".submit"> 

                          <type setby="auto" name="string"/> 

                  </parm>

                  <parm name="userid">

                          <type setby="auto" name="string"/> 

                          <attr setby="auto" name="maxlen" value="16"/> 

                  </parm>

                  <parm name="pin">

                          <type setby="auto" name="string"/> 

                          <attr setby="auto" name="maxlen" value="4"/> 

                  </parm>

                  <parm name="transaction">

                          <type setby="auto" name="string"/> 

                          <attr setby="auto" name="maxlen" value="5"/> 

                  </parm>

          </url>

  </parmguard>

  
 You would then need to translate this mod_parmguard ruleset into modsecurity rules like this -
  
    Userid. For the  userid parameter, we want to restrict the input size to maximum of 16 numerical digits.
 
    PIN. Similar to the  userid data, we want to only allow four numerical digits for a  pin parameter.
 
    Transaction. In this case, the transaction parameter is a fixed character string of "login," so we want to make sure that this does not change. 

  
   <LocationMatch "^/cgi-bin/wm.cgi">

   SecFilterSelective ARG_userid "[0-9]{17,}"
   SecFilterSelective ARG_pin "[0-9]{5,}"
   SecFilterSelective ARG_transaction "!^login$"
  </LocationMatch>


 
While this works, this is still a relatively manual process.  If you have a large website, I would recommend that you try and update the htmlspider.pl PERL code to automatically create modsecurity output rules.  This is a similar concept as the existing snort2modsec.pl script.  
  
 Hopefully this info helps.
  
 -- 
Ryan C. Barnett
Web Application Security Consortium (WASC) Member
CIS Apache Benchmark Project Lead
SANS Instructor: Securing Apache
GCIA, GCFA, GCIH, GSNA, GCUX, GSEC
Author: Preventing Web Attacks with Apache  
  
 
 
 On 5/20/06, kiran k <kir...@ya...> wrote:   Are there any tools which discovers web application from an input URL.
  
 I am looking for a tool which crawls recursively and finds the forms, form fields, server scripts, cookies and hidden fileds. Based on this information I would like to develop policies. If I have this data in xml it would be even better. 
  
  Any quick starting point would be greatly appreciated, if no tools exists. How about any commercial libraries ?
  
 Thanks,

   

---------------------------------
 Ring'em or ping'em. Make PC-to-phone calls as low as 1¢/min  with Yahoo! Messenger with Voice.  






 

			
---------------------------------
Sneak preview the  all-new Yahoo.com. It's not radically different. Just radically better. 
__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com