w3af-develop — dev list for w3af

[W3af-develop] Authenticated scan support for x-access-token

[Jay X. <jay...@ve...>]

Hi,

We are using JWT token after user name/password authentication for the
subsequent http request. The JWT token returned as access-token and the
subsequent request need to include x-aacess-token as part of request.
Otherwise, the server under scan simply rejects http request with 401.

Is this feature being developed or can someone point me to the code where I
can customize myself.

Thanks,

Jay

[W3af-develop] w3af REST API feature requests

[Andres R. <and...@gm...>]

Lists,

    The REST API milestone for w3af is coming to an end, the only
pending feature is "Expose plugin and core (misc|http) configuration"
[0] and OwenTuz is already working on it. Before I move to other
things... any feature requests for the REST API?

[0] https://github.com/andresriancho/w3af/issues/10616

Regards,
-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

Re: [W3af-develop] web service api post method

[冠庭 羅 <bti...@ya...>]

Hi all,

Finally, I can use post method to start scan... 

It spends me almost one week thanks everyone's help.

I tried to figure out how to change ip address and open port... I found it, too!!!

So Thanks again. 


     Andres Riancho <and...@gm...> 於 2015/7/29 (週三) 8:02 PM 寫道﹕
   

 I modified the documentation to show a successful POST request and response:
http://docs.w3af.org/en/latest/api/scans.html#starting-a-scan



On Wed, Jul 29, 2015 at 12:29 AM, 冠庭 羅 <bti...@ya...> wrote:
> Hi,
> I think I got what you mean..
>
> @POST
>    @SuppressWarnings("unchecked")
>    @Path("/ScanTarget")
>    @Consumes(MediaType.APPLICATION_JSON)
>    @Produces(MediaType.APPLICATION_JSON)
>    public void ScanIt() throws JSONException, IOException{
>
>                File _file = new
> File("/home/tiff/w3af/profiles/fast_scan.pw3af");
>
>                JSONObject jsonObject = new JSONObject();
>
>                jsonObject.put("scan_profile",
> connect.convertFileToString(_file));
>
>                JSONArray jsonArray = new JSONArray();
>
>                jsonArray.put(target.get_tagetURL());
>
>                jsonObject.put("target_url", jsonArray);
>
>                System.out.println("output：" + jsonObject.toString());
>
>                ClientConfig config = new DefaultClientConfig();
>
>                Client client = Client.create(config);
>
>                WebResource service =
> client.resource("http://127.0.0.1:5000/scans/");
>
>                ClientResponse client_response =
> service.accept(MediaType.APPLICATION_JSON).post(ClientResponse.class,
> jsonObject);
>    }
>
>    /*Convert file to Base64 String*/
>    public String convertFileToString(File file) throws IOException{
>        byte[] bytes = Files.readAllBytes(file.toPath());
>        return new String(Base64.encode(bytes));
>    }
>
> Although it still has clientHandlerException now.. but is this a correct
> direction what you want to tell me?
>
> Thanks
> Tiff
>
>
>
> Andres Riancho <and...@gm...> 於 2015/7/28 (週二) 7:47 PM 寫道﹕
>
>
> Tiff,
>
>    Why do you create a new email thread for each email you send?
>
> On Tue, Jul 28, 2015 at 12:13 AM, 冠庭 羅 <bti...@ya...> wrote:
>> Hi Andrés Riancho,
>>
>> Sorry.. That is language gap... I thought that filename is like
>> "fast_scan.pw3af" so that I keep trying on folder name
>> But anyway, I tried "../../fast_scan.pw3af" before...
>> my code is like this (only change to the scan_profile)
>>    @POST
>>    @SuppressWarnings("unchecked")
>>    @Path("/ScanTarget")
>>    @Consumes(MediaType.APPLICATION_JSON)
>>    @Produces(MediaType.APPLICATION_JSON)
>>    public int ScanIt() throws IOException {
>>      JSONObject jsonObject = new JSONObject();
>>      jsonObject.put("scan_profile",
>> "/home/tiff/w3af/profiles/fast_scan.pw3af");
>
> Once again, you need to send THE CONTENTS OF THE FILE, NOT THE FILE NAME.
>
>>      JSONArray jsonArray = new JSONArray();
>>      jsonArray.add("http://testaspnet.vulnweb.com/");
>>      jsonObject.put("target_url", jsonArray);
>>      System.out.println("output：" + jsonObject.toJSONString());
>>        connect.set_response(
>>                            connect.get_service().
>>                            request().
>>                            header("Content-Type", "application/json").
>>                        post(Entity.json(jsonObject.toJSONString())));
>>                      return connect.get_status();
>>    }
>>
>> and I got the status 301 message from w3af_api console
>>
>> (venv)[root@localhost w3af]# ./w3af_api
>>  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
>> 127.0.0.1 - - [27/Jul/2015 11:19:40] "POST /scans HTTP/1.1" 301 -
>> 127.0.0.1 - - [27/Jul/2015 11:19:40] "GET /scans/ HTTP/1.1" 200 -
>>
>> What's wrong in my web service or somewhere I lost it?
>>
>> Thanks
>>
>> Tiff
>>
>>
>>
>> ------------------------------------------------------------------------------
>>
>> _______________________________________________
>> W3af-develop mailing list
>> W3a...@li...
>> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>
>>
>
>
>
> --
> Andrés Riancho
>
> Project Leader at w3af - http://w3af.org/
> Web Application Attack and Audit Framework
> Twitter: @w3af
> GPG: 0x93C344F3
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> W3af-develop mailing list
> W3a...@li...
> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>



-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

  

Re: [W3af-develop] web service api post method

[Andres R. <and...@gm...>]

I modified the documentation to show a successful POST request and response:
http://docs.w3af.org/en/latest/api/scans.html#starting-a-scan



On Wed, Jul 29, 2015 at 12:29 AM, 冠庭 羅 <bti...@ya...> wrote:
> Hi,
> I think I got what you mean..
>
> @POST
>     @SuppressWarnings("unchecked")
>     @Path("/ScanTarget")
>     @Consumes(MediaType.APPLICATION_JSON)
>     @Produces(MediaType.APPLICATION_JSON)
>     public void ScanIt() throws JSONException, IOException{
>
>                File _file = new
> File("/home/tiff/w3af/profiles/fast_scan.pw3af");
>
>                JSONObject jsonObject = new JSONObject();
>
>                jsonObject.put("scan_profile",
> connect.convertFileToString(_file));
>
>                JSONArray jsonArray = new JSONArray();
>
>                jsonArray.put(target.get_tagetURL());
>
>                jsonObject.put("target_url", jsonArray);
>
>                System.out.println("output：" + jsonObject.toString());
>
>                ClientConfig config = new DefaultClientConfig();
>
>                Client client = Client.create(config);
>
>                WebResource service =
> client.resource("http://127.0.0.1:5000/scans/");
>
>                ClientResponse client_response =
> service.accept(MediaType.APPLICATION_JSON).post(ClientResponse.class,
> jsonObject);
>    }
>
>     /*Convert file to Base64 String*/
>     public String convertFileToString(File file) throws IOException{
>         byte[] bytes = Files.readAllBytes(file.toPath());
>         return new String(Base64.encode(bytes));
>     }
>
> Although it still has clientHandlerException now.. but is this a correct
> direction what you want to tell me?
>
> Thanks
> Tiff
>
>
>
> Andres Riancho <and...@gm...> 於 2015/7/28 (週二) 7:47 PM 寫道﹕
>
>
> Tiff,
>
>     Why do you create a new email thread for each email you send?
>
> On Tue, Jul 28, 2015 at 12:13 AM, 冠庭 羅 <bti...@ya...> wrote:
>> Hi Andrés Riancho,
>>
>> Sorry.. That is language gap... I thought that filename is like
>> "fast_scan.pw3af" so that I keep trying on folder name
>> But anyway, I tried "../../fast_scan.pw3af" before...
>> my code is like this (only change to the scan_profile)
>>    @POST
>>    @SuppressWarnings("unchecked")
>>    @Path("/ScanTarget")
>>    @Consumes(MediaType.APPLICATION_JSON)
>>    @Produces(MediaType.APPLICATION_JSON)
>>    public int ScanIt() throws IOException {
>>      JSONObject jsonObject = new JSONObject();
>>      jsonObject.put("scan_profile",
>> "/home/tiff/w3af/profiles/fast_scan.pw3af");
>
> Once again, you need to send THE CONTENTS OF THE FILE, NOT THE FILE NAME.
>
>>      JSONArray jsonArray = new JSONArray();
>>      jsonArray.add("http://testaspnet.vulnweb.com/");
>>      jsonObject.put("target_url", jsonArray);
>>      System.out.println("output：" + jsonObject.toJSONString());
>>        connect.set_response(
>>                            connect.get_service().
>>                            request().
>>                            header("Content-Type", "application/json").
>>                        post(Entity.json(jsonObject.toJSONString())));
>>                      return connect.get_status();
>>    }
>>
>> and I got the status 301 message from w3af_api console
>>
>> (venv)[root@localhost w3af]# ./w3af_api
>>  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
>> 127.0.0.1 - - [27/Jul/2015 11:19:40] "POST /scans HTTP/1.1" 301 -
>> 127.0.0.1 - - [27/Jul/2015 11:19:40] "GET /scans/ HTTP/1.1" 200 -
>>
>> What's wrong in my web service or somewhere I lost it?
>>
>> Thanks
>>
>> Tiff
>>
>>
>>
>> ------------------------------------------------------------------------------
>>
>> _______________________________________________
>> W3af-develop mailing list
>> W3a...@li...
>> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>
>>
>
>
>
> --
> Andrés Riancho
>
> Project Leader at w3af - http://w3af.org/
> Web Application Attack and Audit Framework
> Twitter: @w3af
> GPG: 0x93C344F3
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> W3af-develop mailing list
> W3a...@li...
> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>



-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

Re: [W3af-develop] web service api post method

[冠庭 羅 <bti...@ya...>]

Hi, I think I got what you mean..
@POST    @SuppressWarnings("unchecked")    @Path("/ScanTarget")    @Consumes(MediaType.APPLICATION_JSON)    @Produces(MediaType.APPLICATION_JSON)    public void ScanIt() throws JSONException, IOException{
               File _file = new File("/home/tiff/w3af/profiles/fast_scan.pw3af");
               JSONObject jsonObject = new JSONObject();
               jsonObject.put("scan_profile", connect.convertFileToString(_file));
               JSONArray jsonArray = new JSONArray();

               jsonArray.put(target.get_tagetURL());
               jsonObject.put("target_url", jsonArray);
               System.out.println("output：" + jsonObject.toString());
               ClientConfig config = new DefaultClientConfig();
               Client client = Client.create(config);
               WebResource service = client.resource("http://127.0.0.1:5000/scans/");
               ClientResponse client_response = service.accept(MediaType.APPLICATION_JSON).post(ClientResponse.class,                        jsonObject);   }
    /*Convert file to Base64 String*/
    public String convertFileToString(File file) throws IOException{        byte[] bytes = Files.readAllBytes(file.toPath());           return new String(Base64.encode(bytes));    }

Although it still has clientHandlerException now.. but is this a correct direction what you want to tell me?
ThanksTiff 


     Andres Riancho <and...@gm...> 於 2015/7/28 (週二) 7:47 PM 寫道﹕
   

 Tiff,

    Why do you create a new email thread for each email you send?

On Tue, Jul 28, 2015 at 12:13 AM, 冠庭 羅 <bti...@ya...> wrote:
> Hi Andrés Riancho,
>
> Sorry.. That is language gap... I thought that filename is like
> "fast_scan.pw3af" so that I keep trying on folder name
> But anyway, I tried "../../fast_scan.pw3af" before...
> my code is like this (only change to the scan_profile)
>    @POST
>    @SuppressWarnings("unchecked")
>    @Path("/ScanTarget")
>    @Consumes(MediaType.APPLICATION_JSON)
>    @Produces(MediaType.APPLICATION_JSON)
>    public int ScanIt() throws IOException {
>      JSONObject jsonObject = new JSONObject();
>      jsonObject.put("scan_profile",
> "/home/tiff/w3af/profiles/fast_scan.pw3af");

Once again, you need to send THE CONTENTS OF THE FILE, NOT THE FILE NAME.

>      JSONArray jsonArray = new JSONArray();
>      jsonArray.add("http://testaspnet.vulnweb.com/");
>      jsonObject.put("target_url", jsonArray);
>      System.out.println("output：" + jsonObject.toJSONString());
>        connect.set_response(
>                            connect.get_service().
>                            request().
>                            header("Content-Type", "application/json").
>                        post(Entity.json(jsonObject.toJSONString())));
>                      return connect.get_status();
>    }
>
> and I got the status 301 message from w3af_api console
>
> (venv)[root@localhost w3af]# ./w3af_api
>  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
> 127.0.0.1 - - [27/Jul/2015 11:19:40] "POST /scans HTTP/1.1" 301 -
> 127.0.0.1 - - [27/Jul/2015 11:19:40] "GET /scans/ HTTP/1.1" 200 -
>
> What's wrong in my web service or somewhere I lost it?
>
> Thanks
>
> Tiff
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> W3af-develop mailing list
> W3a...@li...
> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>



-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

  

Re: [W3af-develop] web service api post method

[Andres R. <and...@gm...>]

Tiff,

    Why do you create a new email thread for each email you send?

On Tue, Jul 28, 2015 at 12:13 AM, 冠庭 羅 <bti...@ya...> wrote:
> Hi Andrés Riancho,
>
> Sorry.. That is language gap... I thought that filename is like
> "fast_scan.pw3af" so that I keep trying on folder name
> But anyway, I tried "../../fast_scan.pw3af" before...
> my code is like this (only change to the scan_profile)
>    @POST
>    @SuppressWarnings("unchecked")
>    @Path("/ScanTarget")
>    @Consumes(MediaType.APPLICATION_JSON)
>    @Produces(MediaType.APPLICATION_JSON)
>    public int ScanIt() throws IOException {
>      JSONObject jsonObject = new JSONObject();
>      jsonObject.put("scan_profile",
> "/home/tiff/w3af/profiles/fast_scan.pw3af");

Once again, you need to send THE CONTENTS OF THE FILE, NOT THE FILE NAME.

>      JSONArray jsonArray = new JSONArray();
>      jsonArray.add("http://testaspnet.vulnweb.com/");
>      jsonObject.put("target_url", jsonArray);
>      System.out.println("output：" + jsonObject.toJSONString());
>        connect.set_response(
>                             connect.get_service().
>                             request().
>                             header("Content-Type", "application/json").
>                         post(Entity.json(jsonObject.toJSONString())));
>                       return connect.get_status();
>     }
>
> and I got the status 301 message from w3af_api console
>
> (venv)[root@localhost w3af]# ./w3af_api
>  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
> 127.0.0.1 - - [27/Jul/2015 11:19:40] "POST /scans HTTP/1.1" 301 -
> 127.0.0.1 - - [27/Jul/2015 11:19:40] "GET /scans/ HTTP/1.1" 200 -
>
> What's wrong in my web service or somewhere I lost it?
>
> Thanks
>
> Tiff
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> W3af-develop mailing list
> W3a...@li...
> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>



-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

Re: [W3af-develop] web service api post method

[Owen T. <ow...@gm...>]

http://httpstatus.es/301
On 28 Jul 2015 7:33 am, "Owen Tuz" <ow...@gm...> wrote:

> Hi Tiff,
>
> That looks like it is working - a 301 response just means that you are
> being automatically redirected from '/scans' to '/scans/' (with the extra
> slash).
>
> Since you then receive HTTP code 200, it looks like your code is correctly
> following the redirect and sending data to /scans/.
>
> If you can print the JSON response from connect somehow, you should see
> that you have started a scan.
>
> To avoid seeing the 301 response at all, just change your code to POST to
> /scans/.
>
> Best regards,
>
> Owen
> On 28 Jul 2015 4:30 am, "冠庭 羅" <bti...@ya...> wrote:
>
>> Hi Andrés Riancho,
>>
>> Sorry.. That is language gap... I thought that filename is like
>> "fast_scan.pw3af" so that I keep trying on folder name
>> But anyway, I tried "../../fast_scan.pw3af" before...
>> my code is like this (only change to the scan_profile)
>>    @POST
>>    @SuppressWarnings("unchecked")
>>    @Path("/ScanTarget")
>>    @Consumes(MediaType.*APPLICATION_JSON*)
>>    @Produces(MediaType.*APPLICATION_JSON*)
>>    *public* *int* ScanIt() *throws* IOException {
>>      JSONObject jsonObject = *new* JSONObject();
>>      jsonObject.put("scan_profile",
>> "/home/tiff/w3af/profiles/fast_scan.pw3af");
>>      JSONArray jsonArray = *new* JSONArray();
>>      jsonArray.add("http://testaspnet.vulnweb.com/");
>>      jsonObject.put("target_url", jsonArray);
>>      System.*out*.println("output：" + jsonObject.toJSONString());
>>        connect.set_response(
>>                             connect.get_service().
>>
>>                             request().
>>                             header("Content-Type", "application/json").
>>                         post(Entity.*json*(jsonObject.toJSONString())));
>>                       *return* connect.get_status();
>>     }
>>
>> and I got the status 301 message from w3af_api console
>>
>> (venv)[root@localhost w3af]# ./w3af_api
>>  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
>> 127.0.0.1 - - [27/Jul/2015 11:19:40] "POST /scans HTTP/1.1" 301 -
>> 127.0.0.1 - - [27/Jul/2015 11:19:40] "GET /scans/ HTTP/1.1" 200 -
>>
>> What's wrong in my web service or somewhere I lost it?
>>
>> Thanks
>>
>> Tiff
>>
>>
>>
>> ------------------------------------------------------------------------------
>>
>> _______________________________________________
>> W3af-develop mailing list
>> W3a...@li...
>> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>>
>>

Re: [W3af-develop] web service api post method

[Owen T. <ow...@gm...>]

Hi Tiff,

That looks like it is working - a 301 response just means that you are
being automatically redirected from '/scans' to '/scans/' (with the extra
slash).

Since you then receive HTTP code 200, it looks like your code is correctly
following the redirect and sending data to /scans/.

If you can print the JSON response from connect somehow, you should see
that you have started a scan.

To avoid seeing the 301 response at all, just change your code to POST to
/scans/.

Best regards,

Owen
On 28 Jul 2015 4:30 am, "冠庭 羅" <bti...@ya...> wrote:

> Hi Andrés Riancho,
>
> Sorry.. That is language gap... I thought that filename is like
> "fast_scan.pw3af" so that I keep trying on folder name
> But anyway, I tried "../../fast_scan.pw3af" before...
> my code is like this (only change to the scan_profile)
>    @POST
>    @SuppressWarnings("unchecked")
>    @Path("/ScanTarget")
>    @Consumes(MediaType.*APPLICATION_JSON*)
>    @Produces(MediaType.*APPLICATION_JSON*)
>    *public* *int* ScanIt() *throws* IOException {
>      JSONObject jsonObject = *new* JSONObject();
>      jsonObject.put("scan_profile",
> "/home/tiff/w3af/profiles/fast_scan.pw3af");
>      JSONArray jsonArray = *new* JSONArray();
>      jsonArray.add("http://testaspnet.vulnweb.com/");
>      jsonObject.put("target_url", jsonArray);
>      System.*out*.println("output：" + jsonObject.toJSONString());
>        connect.set_response(
>                             connect.get_service().
>
>                             request().
>                             header("Content-Type", "application/json").
>                         post(Entity.*json*(jsonObject.toJSONString())));
>                       *return* connect.get_status();
>     }
>
> and I got the status 301 message from w3af_api console
>
> (venv)[root@localhost w3af]# ./w3af_api
>  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
> 127.0.0.1 - - [27/Jul/2015 11:19:40] "POST /scans HTTP/1.1" 301 -
> 127.0.0.1 - - [27/Jul/2015 11:19:40] "GET /scans/ HTTP/1.1" 200 -
>
> What's wrong in my web service or somewhere I lost it?
>
> Thanks
>
> Tiff
>
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> W3af-develop mailing list
> W3a...@li...
> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>
>

[W3af-develop] web service api post method

[冠庭 羅 <bti...@ya...>]

Hi Andrés Riancho,

Sorry.. That is language gap... I thought that filename is like "fast_scan.pw3af" so that I keep trying on folder nameBut anyway, I tried "../../fast_scan.pw3af" before...
my code is like this (only change to the scan_profile)    @POST   @SuppressWarnings("unchecked")

   @Path("/ScanTarget")

  @Consumes(MediaType.APPLICATION_JSON)

   @Produces(MediaType.APPLICATION_JSON) 

  public int ScanIt() throws IOException {

    JSONObject jsonObject = new JSONObject();  

    jsonObject.put("scan_profile", "/home/tiff/w3af/profiles/fast_scan.pw3af"); 

    JSONArray jsonArray = new JSONArray();

    jsonArray.add("http://testaspnet.vulnweb.com/");

    jsonObject.put("target_url", jsonArray);

    System.out.println("output：" + jsonObject.toJSONString()); 

      connect.set_response(

                           connect.get_service().                                

                           request().

                           header("Content-Type", "application/json").

                       post(Entity.json(jsonObject.toJSONString())));

                      return connect.get_status();

    }


and I got the status 301 message from w3af_api console 

(venv)[root@localhost w3af]# ./w3af_api * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)127.0.0.1 - - [27/Jul/2015 11:19:40] "POST /scans HTTP/1.1" 301 -127.0.0.1 - - [27/Jul/2015 11:19:40] "GET /scans/ HTTP/1.1" 200 -

What's wrong in my web service or somewhere I lost it?

Thanks
Tiff 

Re: [W3af-develop] web service api scans ( post method)

[Andres R. <and...@gm...>]

On Mon, Jul 27, 2015 at 5:38 AM, 冠庭 羅 <bti...@ya...> wrote:
> Hi,
> I have tried it a day, if I write like this
> ---
> @POST
> @SuppressWarnings("unchecked")
> @Path("/ScanTarget")
> @Consumes(MediaType.APPLICATION_JSON)
> @Produces(MediaType.APPLICATION_JSON)
>
> public int ScanIt() throws IOException {
>     JSONObject jsonObject = new JSONObject();
>     jsonObject.put("scan_profile", "/home/tiff/w3af/profiles");
>
>     JSONArray jsonArray = new JSONArray();
>     jsonArray.add("http://testaspnet.vulnweb.com/");
>
>     jsonObject.put("target_url", jsonArray);
>
>     System.out.println("output：" + jsonObject.toJSONString());
>
>     connect.set_response(
>     connect.get_service().
>     request().
>     header("Content-Type", "application/json").
>     post(Entity.json(jsonObject.toJSONString())));
>
>     return connect.get_status();
> }
> ---
> And system log is show like this：
>
> {"scan_profile":"\/home\/tiff\/w3af\/profiles","target_url":["http:\/\/testaspnet.vulnweb.com\/"]}
>
> I think that is quite match with
>
> data = {'scan_profile': file('/path/to/profile.pw3af').read(),
>         'target_urls': ['http://127.0.0.1:8000/audit/sql_injection/']}
>
> Need to put " file(' ... ').read() " into JSONObject?
> I think that is weird and of course I did it, but failure.
> I always get Http status 500...
>
> I don't really understand what happened and don't know how to do only by
> reading "starting a scan" information.
> Sorry about that I'm not good at this.
> Could you give me more details or suggestion about how to start it up?

You need to send the CONTENTS of the profile file, not the filename.

So, instead of "\/home\/tiff\/w3af\/profiles"  (which would be
incorrect anyways) send the result of reading the file
"/home/tiff/w3af/profiles/some_profile.pw3af"

> Thanks a lot.
>
> Tiff
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> W3af-develop mailing list
> W3a...@li...
> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>



-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

[W3af-develop] web service api scans ( post method)

[冠庭 羅 <bti...@ya...>]

Hi, I have tried it a day, if I write like this
---@POST @SuppressWarnings("unchecked") @Path("/ScanTarget")@Consumes(MediaType.APPLICATION_JSON)@Produces(MediaType.APPLICATION_JSON) 
public int ScanIt() throws IOException {      JSONObject jsonObject = new JSONObject();       jsonObject.put("scan_profile", "/home/tiff/w3af/profiles");             JSONArray jsonArray = new JSONArray();    jsonArray.add("http://testaspnet.vulnweb.com/");            jsonObject.put("target_url", jsonArray);        System.out.println("output：" + jsonObject.toJSONString()); 

     connect.set_response(     connect.get_service().          request().     header("Content-Type", "application/json").     post(Entity.json(jsonObject.toJSONString())));
     return connect.get_status();}---And system log is show like this：

{"scan_profile":"\/home\/tiff\/w3af\/profiles","target_url":["http:\/\/testaspnet.vulnweb.com\/"]}

I think that is quite match with data = {'scan_profile': file('/path/to/profile.pw3af').read(),
        'target_urls': ['http://127.0.0.1:8000/audit/sql_injection/']}Need to put " file(' ... ').read() " into JSONObject?
I think that is weird and of course I did it, but failure.I always get Http status 500...

I don't really understand what happened and don't know how to do only by reading "starting a scan" information.Sorry about that I'm not good at this.
Could you give me more details or suggestion about how to start it up?
Thanks a lot.
Tiff

[W3af-develop] web service api scans ( post method)

[冠庭 羅 <bti...@ya...>]

Hi, 

I try it for several times but I still get 405 now. I don't know why..
And this is my code...
    @POST    @Path("/ScanTarget")    @Produces(MediaType.APPLICATION_JSON)     public int ScanIt() {         connect.set_response(         connect.get_service().              request().         header("Content-Type", "application/json").         post(Entity.json("{\"scan_profile\":/home/tiff/w3af/profiles, \"target_urls\" : [\"http://testaspnet.vulnweb.com/\"]}")));        return connect.get_status();    }

I just used href="webapi/ScanTarget" to call it.

Thanks.

Tiff

Re: [W3af-develop] Web service API Questions

[Andres R. <and...@gm...>]

Tiff,

On Thu, Jul 23, 2015 at 9:21 PM, 冠庭 羅 <bti...@ya...> wrote:
> Hi all,
>
> I tried to figure out how to use W3af 's Client API, using Postman to help
> me to run scan by POST method.
>
>      I had already use Content-Type: application/json
>      and in the raw json
>
>      {
>         "scan_profile" : {"scan_profile" :
> "/home/tiff/w3af/profiles/OWASP_TOP10.pw3af"},
>         "target_urls" : {"target_urls" : "http://127.0.0.1:7272"}
>      }

Why do you repeat the scan profile in scan profile? Your request
should look like this one:
http://docs.w3af.org/en/latest/api/scans.html#starting-a-scan


>      ----------------------------------------------
>      I got the error message :
>
>     {
>     "code": 500,
>     "exception_type": "TypeError",
>     "filename": "scans.py",
>     "function_name": "create_temp_profile",
>     "line_number": 51,
>     "message": "expected a character buffer object",
>     "please": "https://github.com/andresriancho/w3af/issues/new"
>     }
>     -----------
>     -----------
>     this is another json code
>     {
>      "scan_profile" : "/home/tiff/w3af/profiles/OWASP_TOP10.pw3af",

It should be the content of that file, not the filename

>      "target_urls" : "http://127.0.0.1:7272"

The value should be a list

>     }
>     --------
>    the result is
>     {
>         "message": "ConfigParser error in profile:
> \"/tmp/1ac23c74-1433-464d-a12a-b4200a9eddb3.pw3af\". Exception: \"File
> contains no section       headers.\\\nfile:
> /tmp/1ac23c74-1433-464d-a12a-b4200a9eddb3.pw3af, line:
> 1\\\nu'/home/tiff/w3af/profiles/OWASP_TOP10.pw3af'\"",
>   "code": 400
>     }
>
>     I just curious about that am I type the correct json? Is that possible
> for me to assign two json object by using Postman?

Never used postman, not sure what it can do

>    (Now I can just using python to start scanning)
> ----
> And another Question is that I tried to use GET method from other devices to
> catch  the result when I done the scans.
> But it didn't work... I use http://192.168.xxx.xxx:5000/scans/0/kb (it looks
> like i need to use the same machine to type
> http://127.0.0.1:5000/scans/0/kb).
> Is that possible for me to use other machine to get the result or not?

Sure, you just need to bind the REST API to that host/port. See the config file:
http://docs.w3af.org/en/latest/api/index.html#config-file-format

>
>
>  Thanks.
>
>  Have a nice day.
>
>  Tiff
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> W3af-develop mailing list
> W3a...@li...
> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>



-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

[W3af-develop] Web service API Questions

[冠庭 羅 <bti...@ya...>]

Hi all,

 I tried tofigure out how to use W3af 's Client API, using Postman to help me to run scanby POST method.

     I had already use Content-Type: application/json 

    and in the raw json 

     {

       "scan_profile" : {"scan_profile" :"/home/tiff/w3af/profiles/OWASP_TOP10.pw3af"},

       "target_urls" : {"target_urls" :"http://127.0.0.1:7272"}

    }

    ----------------------------------------------

    I got the error message : 

   

    {

   "code": 500,

   "exception_type": "TypeError",

   "filename": "scans.py",

   "function_name": "create_temp_profile",

   "line_number": 51,

   "message": "expected a character buffer object",

   "please": "https://github.com/andresriancho/w3af/issues/new"

   }

   -----------    

   -----------
    this is another json code
    {

    "scan_profile" :"/home/tiff/w3af/profiles/OWASP_TOP10.pw3af",

    "target_urls" : "http://127.0.0.1:7272"

   }

   --------

  the result is 

   {

       "message": "ConfigParser error in profile:\"/tmp/1ac23c74-1433-464d-a12a-b4200a9eddb3.pw3af\". Exception:\"File contains no section       headers.\\\nfile:/tmp/1ac23c74-1433-464d-a12a-b4200a9eddb3.pw3af, line:1\\\nu'/home/tiff/w3af/profiles/OWASP_TOP10.pw3af'\"",

 "code": 400

   }

    I just curious about that am I type the correct json? Is that possible for meto assign two json object by using Postman?
   (Now I can just using python to start scanning)

----
And another Question is that I tried to use GET method from other devices to catch  the result when I done the scans.
But it didn't work... I use http://192.168.xxx.xxx:5000/scans/0/kb (it looks like i need to use the same machine to type http://127.0.0.1:5000/scans/0/kb). 
Is that possible for me to use other machine to get the result or not?



 Thanks.

  Have anice day.

  Tiff

Re: [W3af-develop] W3af Rest Api

[Andres R. <and...@gm...>]

Gorantia,

On Mon, Jun 29, 2015 at 11:00 PM, Gorantla sai <ah...@gm...> wrote:
> Hello,
>
>            I'm currently trying to fetch http transaction data sent by w3af
> using w3af Rest Api . According to this doc we can get the http request and
> response data of a transaction of by sending a GET request to
> /scans/<scan-id>/traffic/<traffic-id> , it is also mentioned that traffic-id
> attribute can be accessed by sending a GET request to
> /scans/<scan-id>/kb/<vulnerability-id> ,the docs regarding /kb/ resource
> mention that it returns only the list of vulnerabilities .So according to
> docs we should be only able to fetch http transaction data of vulnerable
> requests .
>
>           But in twitter andres riancho stated that we can read all http
> requests sent from scanner . First of all i would like to know how i can i
> use rest api to fetch http data of scan started w3ag gui

You can't do that via REST API

> , we need scan id
> to get scan details when i started a scan from w3af gui and then sent
> request to /scans/0, /scans/1 , /scans/2 . It seems like we can only get
> scan details of scan only if it is started using rest api

Yep, GUI and REST API are completely different user interfaces and
don't have any method of exchanging data

> . Can anyone
> explain how can we get scan details of scan started from w3af-gui using rest
> api

You can't

>and also how to get  http transactions  details of all the transactions
> not just the vulnerable ones. .

/scans/<scan-id>/traffic/<any-request-id-you-want>

> Thanks,
> Gorantla Sai.
>
> ------------------------------------------------------------------------------
> Don't Limit Your Business. Reach for the Cloud.
> GigeNET's Cloud Solutions provide you with the tools and support that
> you need to offload your IT needs and focus on growing your business.
> Configured For All Businesses. Start Your Cloud Today.
> https://www.gigenetcloud.com/
> _______________________________________________
> W3af-develop mailing list
> W3a...@li...
> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>



-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

[W3af-develop] W3af Rest Api

[Gorantla s. <ah...@gm...>]

Hello,

           I'm currently trying to fetch http transaction data sent by w3af
using w3af Rest Api . According to this
<http://docs.w3af.org/en/latest/api/traffic.html> doc we can get the http
request and response data of a transaction of by sending a GET request
to  /scans/<scan-id>/traffic/<traffic-id>
, it is also mentioned that traffic-id attribute can be accessed by sending
a GET request to  /scans/<scan-id>/kb/<vulnerability-id> ,the docs
<http://docs.w3af.org/en/latest/api/kb.html> regarding /kb/ resource
mention that it returns only the list of vulnerabilities .So according to
docs we should be only able to fetch http transaction data of vulnerable
requests .

          But in twitter andres riancho stated
<https://twitter.com/w3af/status/615570631204192256> that we can read all
http requests sent from scanner . First of all i would like to know how i
can i use rest api to fetch http data of scan started w3ag gui , we need
scan id to get scan details when i started a scan from w3af gui and then
sent request to /scans/0, /scans/1 , /scans/2 . It seems like we can only
get scan details of scan only if it is started using rest api . Can anyone
explain how can we get scan details of scan started from w3af-gui using
rest api and also how to get  http transactions  details of all the
transactions not just the vulnerable ones. .

Thanks,
Gorantla Sai.

Re: [W3af-develop] [SPAM] celery issues

[Andres R. <and...@gm...>]

https://github.com/andresriancho/w3af/blob/master/w3af/core/data/kb/kb_observer.py
https://github.com/andresriancho/w3af/blob/master/w3af/core/data/kb/knowledge_base.py#L513

On Wed, Apr 22, 2015 at 1:08 PM, Electric Mind <ma...@ze...> wrote:
> Hi, Andres!
>
> Check the knowledge base hooks, I would rather use that instead of
> waiting for the scan to finish. With the hooks you can give the users
> instant feedback by sending a new task to the broker containing the
> vulnerability. Then a worker would consume the task (with the vuln)
> and show it to the user.
>
> <<
>
> You mean just to override Plugin method `kb_append` for example  ?
>
>
> Regards.
>
>
>> On 03 Apr 2015, at 19:08, Andres Riancho <and...@gm...> wrote:
>>
>> Well, yeah, that might be a problem. I recommend:
>>
>> * You should use different processes (prefork/processes should be ok)
>> * Make sure ALL your w3af imports are done INSIDE the celery task.
>> So, this won't work:
>>
>> from w3af import something
>>
>> class Scan(Task):
>>    def run(...):
>>        something()
>>
>> But this might/should:
>>
>> class Scan(Task):
>>    def run(...):
>>        from w3af import something
>>        something()
>>
>>
>> I haven't played with celery+latest w3af, but I'm 100% sure that old
>> w3af versions (that didn't make use of multiprocessing) did work
>> inside celery tasks. Let me know how your implementation goes,
>>
>> On Fri, Apr 3, 2015 at 6:10 AM, Electric Mind <ma...@ze...> wrote:
>>> Hi, Andres.
>>>
>>> Could you please show code example how to start your multiply w3af core
>>> inside of celery processes / prefork.
>>> Example:
>>>
>>> celery -A scanner worker --concurrency 2 -l debug -P prefork.
>>> celery -A scanner worker --concurrency 2 -l debug -P processes
>>>
>>> I have troubles with celery threads in my previous code example, because
>>> they use global Knowledge base object which is shared during the celery
>>> worker live.
>>>
>>> Regards.
>>>
>>> On 25 Mar 2015, at 21:55, Andres Riancho <and...@gm...> wrote:
>>>
>>> Electric Mind,
>>>
>>>   Please read inline,
>>>
>>> On Wed, Mar 25, 2015 at 3:49 PM, Electric Mind <ma...@ze...>
>>> wrote:
>>>
>>> Hello everyone!
>>>
>>> I’d like to start my w3af instances inside of the celery.
>>> Code example is below. What is the best practice for doing that ?
>>>
>>>
>>> First I would recommend you configure celery to:
>>> * Prefetch only one task
>>> * Run one task at the time
>>> * Run scan tasks in different processes
>>>
>>> Regards!
>>>
>>>
>>> scanner.py
>>> -----
>>>
>>> # -*- coding: utf-8 -*-
>>> # !/usr/bin/env python
>>>
>>> from __future__ import absolute_import
>>> from multiprocessing import cpu_count
>>> from w3af.core.controllers.w3afCore import w3afCore
>>> from w3af.plugins.tests.helper import create_target_option_list
>>> from w3af.core.data.parsers.url import URL
>>> from w3af.core.data.options.option_list import OptionList
>>> from w3af.core.data.options.opt_factory import opt_factory
>>> from w3af.core.data.kb import knowledge_base
>>> from w3af.core.data.kb.info_set import InfoSet
>>>
>>> from scanner.celery import app
>>>
>>>
>>> @app.task
>>> def start_scan(host):
>>>   target_opts = create_target_option_list(URL(host))
>>>   core = w3afCore()
>>>   core.WORKER_THREADS = cpu_count() * 2
>>>
>>>
>>> Threads are not processes
>>>
>>>   core.target.set_options(target_opts)
>>>
>>>   core.plugins.set_plugins(
>>>       [
>>>           'xss_protection_header',
>>>           'csp',
>>>           'strange_headers',
>>>           'click_jacking',
>>>           'content_type_options_header'
>>>       ], 'grep')
>>>
>>>   core.plugins.set_plugins(
>>>       [
>>>           'allowed_methods',
>>>           'find_vhosts'
>>>       ], 'infrastructure')
>>>
>>>   core.plugins.set_plugins(
>>>       [
>>>           'dir_file_bruter',
>>>           'robots_txt',
>>>           'ria_enumerator'
>>>       ], 'crawl')
>>>
>>>   core.plugins.set_plugins(
>>>       [
>>>           'ssl',
>>>       ], 'audit')
>>>
>>>   core.plugins.set_plugins(
>>>       [
>>>           'console'
>>>       ], 'output')
>>>   console_options = OptionList()
>>>   console_options.add(opt_factory('verbose', True, 'desc', 'boolean'))
>>>   core.plugins.set_plugin_options('output', 'console', console_options)
>>>
>>>   core.plugins.init_plugins()
>>>   core.start()
>>>
>>>   result = {'information_disclosures': [], 'vulnerabilities': []}
>>>
>>>   for information_disclosure in knowledge_base.kb.get_all_infos():
>>>       if isinstance(information_disclosure, InfoSet):
>>>
>>> result['information_disclosures'].append("{}".format(information_disclosure.get_desc()))
>>>       else:
>>>
>>> result['information_disclosures'].append("{}".format(information_disclosure))
>>>
>>>   for vulnerability in knowledge_base.kb.get_all_vulns():
>>>       result['vulnerabilities'].append("{}".format(vulnerability))
>>>
>>>
>>> Check the knowledge base hooks, I would rather use that instead of
>>> waiting for the scan to finish. With the hooks you can give the users
>>> instant feedback by sending a new task to the broker containing the
>>> vulnerability. Then a worker would consume the task (with the vuln)
>>> and show it to the user.
>>>
>>>   return result
>>>
>>> ------------------------------------------------------------------------------
>>> Dive into the World of Parallel Programming The Go Parallel Website,
>>> sponsored
>>> by Intel and developed in partnership with Slashdot Media, is your hub for
>>> all
>>> things parallel software development, from weekly thought leadership blogs
>>> to
>>> news, videos, case studies, tutorials and more. Take a look and join the
>>> conversation now. http://goparallel.sourceforge.net/
>>> _______________________________________________
>>> W3af-develop mailing list
>>> W3a...@li...
>>> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>>>
>>>
>>>
>>>
>>> --
>>> Andrés Riancho
>>> Project Leader at w3af - http://w3af.org/
>>> Web Application Attack and Audit Framework
>>> Twitter: @w3af
>>> GPG: 0x93C344F3
>>>
>>>
>>
>>
>>
>> --
>> Andrés Riancho
>> Project Leader at w3af - http://w3af.org/
>> Web Application Attack and Audit Framework
>> Twitter: @w3af
>> GPG: 0x93C344F3
>



-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

Re: [W3af-develop] [SPAM] celery issues

[Andres R. <and...@gm...>]

Well, yeah, that might be a problem. I recommend:

 * You should use different processes (prefork/processes should be ok)
 * Make sure ALL your w3af imports are done INSIDE the celery task.
So, this won't work:

from w3af import something

class Scan(Task):
    def run(...):
        something()

But this might/should:

class Scan(Task):
    def run(...):
        from w3af import something
        something()


I haven't played with celery+latest w3af, but I'm 100% sure that old
w3af versions (that didn't make use of multiprocessing) did work
inside celery tasks. Let me know how your implementation goes,

On Fri, Apr 3, 2015 at 6:10 AM, Electric Mind <ma...@ze...> wrote:
> Hi, Andres.
>
> Could you please show code example how to start your multiply w3af core
> inside of celery processes / prefork.
> Example:
>
> celery -A scanner worker --concurrency 2 -l debug -P prefork.
> celery -A scanner worker --concurrency 2 -l debug -P processes
>
> I have troubles with celery threads in my previous code example, because
> they use global Knowledge base object which is shared during the celery
> worker live.
>
> Regards.
>
> On 25 Mar 2015, at 21:55, Andres Riancho <and...@gm...> wrote:
>
> Electric Mind,
>
>    Please read inline,
>
> On Wed, Mar 25, 2015 at 3:49 PM, Electric Mind <ma...@ze...>
> wrote:
>
> Hello everyone!
>
> I’d like to start my w3af instances inside of the celery.
> Code example is below. What is the best practice for doing that ?
>
>
> First I would recommend you configure celery to:
> * Prefetch only one task
> * Run one task at the time
> * Run scan tasks in different processes
>
> Regards!
>
>
> scanner.py
> -----
>
> # -*- coding: utf-8 -*-
> # !/usr/bin/env python
>
> from __future__ import absolute_import
> from multiprocessing import cpu_count
> from w3af.core.controllers.w3afCore import w3afCore
> from w3af.plugins.tests.helper import create_target_option_list
> from w3af.core.data.parsers.url import URL
> from w3af.core.data.options.option_list import OptionList
> from w3af.core.data.options.opt_factory import opt_factory
> from w3af.core.data.kb import knowledge_base
> from w3af.core.data.kb.info_set import InfoSet
>
> from scanner.celery import app
>
>
> @app.task
> def start_scan(host):
>    target_opts = create_target_option_list(URL(host))
>    core = w3afCore()
>    core.WORKER_THREADS = cpu_count() * 2
>
>
> Threads are not processes
>
>    core.target.set_options(target_opts)
>
>    core.plugins.set_plugins(
>        [
>            'xss_protection_header',
>            'csp',
>            'strange_headers',
>            'click_jacking',
>            'content_type_options_header'
>        ], 'grep')
>
>    core.plugins.set_plugins(
>        [
>            'allowed_methods',
>            'find_vhosts'
>        ], 'infrastructure')
>
>    core.plugins.set_plugins(
>        [
>            'dir_file_bruter',
>            'robots_txt',
>            'ria_enumerator'
>        ], 'crawl')
>
>    core.plugins.set_plugins(
>        [
>            'ssl',
>        ], 'audit')
>
>    core.plugins.set_plugins(
>        [
>            'console'
>        ], 'output')
>    console_options = OptionList()
>    console_options.add(opt_factory('verbose', True, 'desc', 'boolean'))
>    core.plugins.set_plugin_options('output', 'console', console_options)
>
>    core.plugins.init_plugins()
>    core.start()
>
>    result = {'information_disclosures': [], 'vulnerabilities': []}
>
>    for information_disclosure in knowledge_base.kb.get_all_infos():
>        if isinstance(information_disclosure, InfoSet):
>
> result['information_disclosures'].append("{}".format(information_disclosure.get_desc()))
>        else:
>
> result['information_disclosures'].append("{}".format(information_disclosure))
>
>    for vulnerability in knowledge_base.kb.get_all_vulns():
>        result['vulnerabilities'].append("{}".format(vulnerability))
>
>
> Check the knowledge base hooks, I would rather use that instead of
> waiting for the scan to finish. With the hooks you can give the users
> instant feedback by sending a new task to the broker containing the
> vulnerability. Then a worker would consume the task (with the vuln)
> and show it to the user.
>
>    return result
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> W3af-develop mailing list
> W3a...@li...
> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>
>
>
>
> --
> Andrés Riancho
> Project Leader at w3af - http://w3af.org/
> Web Application Attack and Audit Framework
> Twitter: @w3af
> GPG: 0x93C344F3
>
>



-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

[W3af-develop] Changes in XML output

[Andres R. <and...@gm...>]

List,

    I'm working on extending w3af's features by providing long
descriptions for vulnerabilities [1], which lead to a change in the
XML output format [0]. All tools that parse w3af's output need to be
updated to handle this new format. XSD file is here [2]

    These changes are, for now, in a feature branch and don't affect
the latest w3af (which comes from master branch). Be prepared for a
merge in <7 days!

[0] https://github.com/andresriancho/w3af/commit/262ebd235f43a6d3e6f568e0346d7d83b3ffe113
[1] https://github.com/andresriancho/w3af/issues/9174
[2] https://github.com/andresriancho/w3af/blob/262ebd235f43a6d3e6f568e0346d7d83b3ffe113/w3af/plugins/output/xml_file/report.xsd

Regards,
-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

Re: [W3af-develop] [SPAM] celery issues

[Andres R. <and...@gm...>]

Electric Mind,

    Please read inline,

On Wed, Mar 25, 2015 at 3:49 PM, Electric Mind <ma...@ze...> wrote:
> Hello everyone!
>
> I’d like to start my w3af instances inside of the celery.
> Code example is below. What is the best practice for doing that ?

First I would recommend you configure celery to:
 * Prefetch only one task
 * Run one task at the time
 * Run scan tasks in different processes

> Regards!
>
>
> scanner.py
> -----
>
> # -*- coding: utf-8 -*-
> # !/usr/bin/env python
>
> from __future__ import absolute_import
> from multiprocessing import cpu_count
> from w3af.core.controllers.w3afCore import w3afCore
> from w3af.plugins.tests.helper import create_target_option_list
> from w3af.core.data.parsers.url import URL
> from w3af.core.data.options.option_list import OptionList
> from w3af.core.data.options.opt_factory import opt_factory
> from w3af.core.data.kb import knowledge_base
> from w3af.core.data.kb.info_set import InfoSet
>
> from scanner.celery import app
>
>
> @app.task
> def start_scan(host):
>     target_opts = create_target_option_list(URL(host))
>     core = w3afCore()
>     core.WORKER_THREADS = cpu_count() * 2

Threads are not processes

>     core.target.set_options(target_opts)
>
>     core.plugins.set_plugins(
>         [
>             'xss_protection_header',
>             'csp',
>             'strange_headers',
>             'click_jacking',
>             'content_type_options_header'
>         ], 'grep')
>
>     core.plugins.set_plugins(
>         [
>             'allowed_methods',
>             'find_vhosts'
>         ], 'infrastructure')
>
>     core.plugins.set_plugins(
>         [
>             'dir_file_bruter',
>             'robots_txt',
>             'ria_enumerator'
>         ], 'crawl')
>
>     core.plugins.set_plugins(
>         [
>             'ssl',
>         ], 'audit')
>
>     core.plugins.set_plugins(
>         [
>             'console'
>         ], 'output')
>     console_options = OptionList()
>     console_options.add(opt_factory('verbose', True, 'desc', 'boolean'))
>     core.plugins.set_plugin_options('output', 'console', console_options)
>
>     core.plugins.init_plugins()
>     core.start()
>
>     result = {'information_disclosures': [], 'vulnerabilities': []}
>
>     for information_disclosure in knowledge_base.kb.get_all_infos():
>         if isinstance(information_disclosure, InfoSet):
>
> result['information_disclosures'].append("{}".format(information_disclosure.get_desc()))
>         else:
>
> result['information_disclosures'].append("{}".format(information_disclosure))
>
>     for vulnerability in knowledge_base.kb.get_all_vulns():
>         result['vulnerabilities'].append("{}".format(vulnerability))

Check the knowledge base hooks, I would rather use that instead of
waiting for the scan to finish. With the hooks you can give the users
instant feedback by sending a new task to the broker containing the
vulnerability. Then a worker would consume the task (with the vuln)
and show it to the user.

>     return result
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> W3af-develop mailing list
> W3a...@li...
> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>



-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

Re: [W3af-develop] Continuous integration to debian packages

[Andres R. <and...@gm...>]

I spent the last 15 working hours on trying to build a new .deb
package for Kali + automate some of the testing that's required after
building a package. More information about it here:

https://github.com/andresriancho/w3af-kali#testing-the-deb-files

This was done in an effort to release 1.6.46 in Kali and, while it's
related, is not focused on automating the whole process (which needs
to be done at https://github.com/andresriancho/w3af/issues/3351)

On Mon, Mar 2, 2015 at 1:11 PM, Andres Riancho <and...@gm...> wrote:
> Sergey,
>
> On Mon, Mar 2, 2015 at 9:42 AM, Sergey <w3...@ko...> wrote:
>> Thank you, Andres.
>>
>> I'm not sure that w3af-kali fits my needs but it's definitely worth to look
>> at it.
>
> Well, Kali uses Debian's packaging system to the work that's done for
> Kali can be re-used in Debian. The bad thing about Debian is that it
> has very strict policies on what get's into their repositories, which
> makes it a lengthy process (it took Luciano Bello and I almost 8
> months to get w3af in the first time). Kali devs makes it
> easier/faster :)
>
>> As for the ticket, how much it would take to resolve it?
>
> If you're a packaging master, it might take you around 8 and 12h.
>
>> What are the main problems?
>
> I believe there are no known issues/limitations/problems at sight,
> just reviewed the ticket and it seems all straight forward. The main
> thing here is to build all the scripts that circleci.com will use
> during the build process and make sure that they:
>
>  * Work
>  * Give us good log messages to debug any issues
>  * Are stable (10 consecutive builds all PASS)
>
>> And btw how are dependencies' for w3af resolved in kali? Looks like package
>> may become broken by some unanticipated update of w3af requirements.
>
> Yep, that's something that will happen and there is no automated fix.
> When that happens the w3af-kali build must break and we need to:
>
>  a) Add the (existing in Kali) dependency to debian/control Depends
>  b) Contact the Kali devs for them to add the required dependencies
> (if they are not in Kali), and then goto a)
>
> See https://github.com/andresriancho/w3af-kali#package-dependencies
>
> I believe the first steps would be for you to clone the w3af-kali
> repository, sign up for circleci (free), hook w3af-kali and circleci,
> and then start to play around with the build scripts that will
> ultimately create and test the .deb file.
>
>>
>> On 27.02.2015 19:09, Andres Riancho wrote:
>>>
>>> Guys,
>>>
>>>      Just found a github ticket you might find interesting.
>>>      https://github.com/andresriancho/w3af/issues/3351
>>>
>>> On Thu, Feb 26, 2015 at 10:37 AM, Nich Ramsey <oni...@gm...> wrote:
>>>>
>>>> Hi Andres,
>>>>
>>>> I just started building the packages with virtualenv on a local vm.
>>>> On Feb 26, 2015 2:48 AM, "Andres Riancho" <and...@gm...>
>>>> wrote:
>>>>>
>>>>>
>>>>> Sergey,
>>>>>
>>>>> On Thu, Feb 26, 2015 at 1:30 AM, Sergey <w3...@ko...> wrote:
>>>>>>
>>>>>> Hi, Andres and everybody.
>>>>>>
>>>>>> Right now I see that we have working CI builds of w3af Docker images.
>>>>>
>>>>>
>>>>> We do! I've been working on the docker images last week and you can
>>>>> see the latest in the develop branch :)
>>>>>
>>>>>> I'd like to know if anybody has some setup for building of w3af debian
>>>>>> packages. For example using virtualenv/dh-virtualenv or fabric/robe or
>>>>>> something like this?
>>>>>
>>>>>
>>>>> Not that I know of, but you might be interested in this email thread
>>>>> [0] where we discuss building kali packages in an automated way. The
>>>>> summary is:
>>>>>   * I would love to have automated builds of .deb
>>>>>   * We could use docker images for testing the created deb packages in
>>>>> Debian/Kali/etc.
>>>>>   * I've been using circleci.com and would like to continue using that
>>>>> CI system (free for open source)
>>>>>   * This repository is the closest thing we have to an automated .deb
>>>>> package [1] build
>>>>>
>>>>> If you want to help, let me know and we can draft a plan.
>>>>>
>>>>> [0]
>>>>>
>>>>> http://sourceforge.net/p/w3af/mailman/w3af-develop/thread/CA%2B1Rt66cek7ubXJHYe%2BbYxbUZg1HyvRDH7DViQkbUbvbWCxPLA%40mail.gmail.com/
>>>>> [1] https://github.com/andresriancho/w3af-kali/
>>>>>
>>>> I haven't worked with circleci before, but I would be willing to go along
>>>> with Sergey in this if he wants to. If he wanted to tackle the circleci
>>>> alone, I could just share what I learn from the virtualenv build process.
>>>>
>>>>
>>>>>> Thank you.
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Dive into the World of Parallel Programming The Go Parallel Website,
>>>>>> sponsored
>>>>>> by Intel and developed in partnership with Slashdot Media, is your hub
>>>>>> for all
>>>>>> things parallel software development, from weekly thought leadership
>>>>>> blogs to
>>>>>> news, videos, case studies, tutorials and more. Take a look and join
>>>>>> the
>>>>>> conversation now. http://goparallel.sourceforge.net/
>>>>>> _______________________________________________
>>>>>> W3af-develop mailing list
>>>>>> W3a...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Andrés Riancho
>>>>> Project Leader at w3af - http://w3af.org/
>>>>> Web Application Attack and Audit Framework
>>>>> Twitter: @w3af
>>>>> GPG: 0x93C344F3
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Dive into the World of Parallel Programming The Go Parallel Website,
>>>>> sponsored
>>>>> by Intel and developed in partnership with Slashdot Media, is your hub
>>>>> for
>>>>> all
>>>>> things parallel software development, from weekly thought leadership
>>>>> blogs
>>>>> to
>>>>> news, videos, case studies, tutorials and more. Take a look and join the
>>>>> conversation now. http://goparallel.sourceforge.net/
>>>>> _______________________________________________
>>>>> W3af-develop mailing list
>>>>> W3a...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>>>
>>>
>>>
>>>
>>
>
>
>
> --
> Andrés Riancho
> Project Leader at w3af - http://w3af.org/
> Web Application Attack and Audit Framework
> Twitter: @w3af
> GPG: 0x93C344F3



-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

Re: [W3af-develop] w3af q

[Andres R. <and...@gm...>]

Electric Mind,

On Tue, Mar 3, 2015 at 5:40 AM, Electric Mind <ma...@ze...> wrote:
> Hello everyone!
>
> Here is my code sample:
>
> #!/usr/bin/env python
>
> from w3af.core.controllers.w3afCore import w3afCore
> from w3af.plugins.tests.helper import create_target_option_list
> from w3af.core.data.parsers.url import URL
> from w3af.core.data.options.option_list import OptionList
> from w3af.core.data.options.opt_factory import opt_factory
> from  w3af.core.data.kb import knowledge_base
>
> w3afcore = w3afCore()
> target_opts = create_target_option_list(URL("http://localhost"))
> w3afcore.target.set_options(target_opts)
>
> w3afcore.plugins.set_plugins(['click_jacking'] , 'grep')
>
> w3afcore.plugins.init_plugins()
> w3afcore.start()
>
>
> I’d like to understand where is method “OPTIONS * HTTP/1.0” get executed. I
> couldn’t find it in fingerprint_404 class...
>
> Below you could see my apache logs:
>
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428
> "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0;
> w3af.org)”
>
> ==> error.log <==
> [Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist:
> /var/www/lvX119D7.cgi
> [Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist:
> /var/www/BdfTExEg.do
>
> ==> other_vhosts_access.log <==
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /lvX119D7.cgi
> HTTP/1.1" 404 447 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
> Trident/4.0; w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /BdfTExEg.do
> HTTP/1.1" 404 446 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
> Trident/4.0; w3af.org)"
>
> ==> error.log <==
>
> [Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist:
> /var/www/zneaLKli.asp
> [Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist:
> /var/www/nj2Wub66.foobar
> [Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist:
> /var/www/mp5JzKQz.htm
> [Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist:
> /var/www/Ll5OAx0X.py
> [Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist:
> /var/www/VCVM34f7.gif
> [Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist:
> /var/www/348kxmyI.htmls
> [Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist:
> /var/www/lY3gD4if.jsp
> [Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist:
> /var/www/FJZ3bNfK.rb
>
> ==> other_vhosts_access.log <==
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /zneaLKli.asp
> HTTP/1.1" 404 446 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
> Trident/4.0; w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /nj2Wub66.foobar
> HTTP/1.1" 404 450 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
> Trident/4.0; w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /mp5JzKQz.htm
> HTTP/1.1" 404 448 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
> Trident/4.0; w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /Ll5OAx0X.py
> HTTP/1.1" 404 447 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
> Trident/4.0; w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /VCVM34f7.gif
> HTTP/1.1" 404 449 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
> Trident/4.0; w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /348kxmyI.htmls
> HTTP/1.1" 404 448 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
> Trident/4.0; w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /lY3gD4if.jsp
> HTTP/1.1" 404 448 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
> Trident/4.0; w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /FJZ3bNfK.rb
> HTTP/1.1" 404 448 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
> Trident/4.0; w3af.org)"
>
> ==> error.log <==
> [Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist:
> /var/www/JxX8yBjT.xhtml
>
> ==> other_vhosts_access.log <==
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /JxX8yBjT.xhtml
> HTTP/1.1" 404 449 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
> Trident/4.0; w3af.org)"
>
> ==> error.log <==
> [Mon Mar 02 03:46:45 2015] [error] [client ::1] script
> '/var/www/joHWRGWy.php' not found or unable to stat
> [Mon Mar 02 03:46:45 2015] [error] [client ::1] File does not exist:
> /var/www/GFDhaqJu.aspx
>
> ==> other_vhosts_access.log <==
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:45 -0500] "GET /joHWRGWy.php
> HTTP/1.1" 404 448 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
> Trident/4.0; w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:45 -0500] "GET /GFDhaqJu.aspx
> HTTP/1.1" 404 448 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
> Trident/4.0; w3af.org)"
>
> ==> error.log <==
> [Mon Mar 02 03:46:46 2015] [error] [client ::1] File does not exist:
> /var/www/1Qy6y9dj.pl
>
> ==> other_vhosts_access.log <==
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:46 -0500] "GET /1Qy6y9dj.pl
> HTTP/1.1" 404 447 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
> Trident/4.0; w3af.org)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:47 -0500] "OPTIONS * HTTP/1.0" 200
> 126 "-" "Apache/2.2.22 (Debian) (internal dummy connection)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:48 -0500] "OPTIONS * HTTP/1.0" 200
> 126 "-" "Apache/2.2.22 (Debian) (internal dummy connection)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:49 -0500] "OPTIONS * HTTP/1.0" 200
> 126 "-" "Apache/2.2.22 (Debian) (internal dummy connection)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:50 -0500] "OPTIONS * HTTP/1.0" 200
> 126 "-" "Apache/2.2.22 (Debian) (internal dummy connection)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:51 -0500] "OPTIONS * HTTP/1.0" 200
> 126 "-" "Apache/2.2.22 (Debian) (internal dummy connection)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:52 -0500] "OPTIONS * HTTP/1.0" 200
> 126 "-" "Apache/2.2.22 (Debian) (internal dummy connection)"
> 127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:53 -0500] "OPTIONS * HTTP/1.0" 200
> 126 "-" "Apache/2.2.22 (Debian) (internal dummy connection)”

w3af doesn't send those HTTP requests, those are for internal
management of child processes in apache, see:
    https://wiki.apache.org/httpd/InternalDummyConnection

>
> Thanks!
>
>
> Best Regards.
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> W3af-develop mailing list
> W3a...@li...
> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>



-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

[W3af-develop] w3af q

[Electric M. <ma...@ze...>]

Hello everyone!

Here is my code sample:

#!/usr/bin/env python

from w3af.core.controllers.w3afCore import w3afCore
from w3af.plugins.tests.helper import create_target_option_list
from w3af.core.data.parsers.url import URL
from w3af.core.data.options.option_list import OptionList
from w3af.core.data.options.opt_factory import opt_factory
from  w3af.core.data.kb import knowledge_base

w3afcore = w3afCore()
target_opts = create_target_option_list(URL("http://localhost <http://localhost/>"))
w3afcore.target.set_options(target_opts)

w3afcore.plugins.set_plugins(['click_jacking'] , 'grep')

w3afcore.plugins.init_plugins()
w3afcore.start()


I’d like to understand where is method “OPTIONS * HTTP/1.0” get executed. I couldn’t find it in fingerprint_404 class...

Below you could see my apache logs:

127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET / HTTP/1.1" 200 428 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)”

==> error.log <==
[Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist: /var/www/lvX119D7.cgi
[Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist: /var/www/BdfTExEg.do

==> other_vhosts_access.log <==
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /lvX119D7.cgi HTTP/1.1" 404 447 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /BdfTExEg.do HTTP/1.1" 404 446 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"

==> error.log <==

[Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist: /var/www/zneaLKli.asp
[Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist: /var/www/nj2Wub66.foobar
[Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist: /var/www/mp5JzKQz.htm
[Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist: /var/www/Ll5OAx0X.py
[Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist: /var/www/VCVM34f7.gif
[Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist: /var/www/348kxmyI.htmls
[Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist: /var/www/lY3gD4if.jsp
[Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist: /var/www/FJZ3bNfK.rb

==> other_vhosts_access.log <==
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /zneaLKli.asp HTTP/1.1" 404 446 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /nj2Wub66.foobar HTTP/1.1" 404 450 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /mp5JzKQz.htm HTTP/1.1" 404 448 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /Ll5OAx0X.py HTTP/1.1" 404 447 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /VCVM34f7.gif HTTP/1.1" 404 449 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /348kxmyI.htmls HTTP/1.1" 404 448 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /lY3gD4if.jsp HTTP/1.1" 404 448 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /FJZ3bNfK.rb HTTP/1.1" 404 448 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"

==> error.log <==
[Mon Mar 02 03:46:44 2015] [error] [client ::1] File does not exist: /var/www/JxX8yBjT.xhtml

==> other_vhosts_access.log <==
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:44 -0500] "GET /JxX8yBjT.xhtml HTTP/1.1" 404 449 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"

==> error.log <==
[Mon Mar 02 03:46:45 2015] [error] [client ::1] script '/var/www/joHWRGWy.php' not found or unable to stat
[Mon Mar 02 03:46:45 2015] [error] [client ::1] File does not exist: /var/www/GFDhaqJu.aspx

==> other_vhosts_access.log <==
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:45 -0500] "GET /joHWRGWy.php HTTP/1.1" 404 448 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:45 -0500] "GET /GFDhaqJu.aspx HTTP/1.1" 404 448 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"

==> error.log <==
[Mon Mar 02 03:46:46 2015] [error] [client ::1] File does not exist: /var/www/1Qy6y9dj.pl

==> other_vhosts_access.log <==
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:46 -0500] "GET /1Qy6y9dj.pl HTTP/1.1" 404 447 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; w3af.org <http://w3af.org/>)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:47 -0500] "OPTIONS * HTTP/1.0" 200 126 "-" "Apache/2.2.22 (Debian) (internal dummy connection)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:48 -0500] "OPTIONS * HTTP/1.0" 200 126 "-" "Apache/2.2.22 (Debian) (internal dummy connection)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:49 -0500] "OPTIONS * HTTP/1.0" 200 126 "-" "Apache/2.2.22 (Debian) (internal dummy connection)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:50 -0500] "OPTIONS * HTTP/1.0" 200 126 "-" "Apache/2.2.22 (Debian) (internal dummy connection)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:51 -0500] "OPTIONS * HTTP/1.0" 200 126 "-" "Apache/2.2.22 (Debian) (internal dummy connection)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:52 -0500] "OPTIONS * HTTP/1.0" 200 126 "-" "Apache/2.2.22 (Debian) (internal dummy connection)"
127.0.1.1:80 ::1 - - [02/Mar/2015:03:46:53 -0500] "OPTIONS * HTTP/1.0" 200 126 "-" "Apache/2.2.22 (Debian) (internal dummy connection)”


Thanks!


Best Regards.

Re: [W3af-develop] Continuous integration to debian packages

[Andres R. <and...@gm...>]

Sergey,

On Mon, Mar 2, 2015 at 9:42 AM, Sergey <w3...@ko...> wrote:
> Thank you, Andres.
>
> I'm not sure that w3af-kali fits my needs but it's definitely worth to look
> at it.

Well, Kali uses Debian's packaging system to the work that's done for
Kali can be re-used in Debian. The bad thing about Debian is that it
has very strict policies on what get's into their repositories, which
makes it a lengthy process (it took Luciano Bello and I almost 8
months to get w3af in the first time). Kali devs makes it
easier/faster :)

> As for the ticket, how much it would take to resolve it?

If you're a packaging master, it might take you around 8 and 12h.

> What are the main problems?

I believe there are no known issues/limitations/problems at sight,
just reviewed the ticket and it seems all straight forward. The main
thing here is to build all the scripts that circleci.com will use
during the build process and make sure that they:

 * Work
 * Give us good log messages to debug any issues
 * Are stable (10 consecutive builds all PASS)

> And btw how are dependencies' for w3af resolved in kali? Looks like package
> may become broken by some unanticipated update of w3af requirements.

Yep, that's something that will happen and there is no automated fix.
When that happens the w3af-kali build must break and we need to:

 a) Add the (existing in Kali) dependency to debian/control Depends
 b) Contact the Kali devs for them to add the required dependencies
(if they are not in Kali), and then goto a)

See https://github.com/andresriancho/w3af-kali#package-dependencies

I believe the first steps would be for you to clone the w3af-kali
repository, sign up for circleci (free), hook w3af-kali and circleci,
and then start to play around with the build scripts that will
ultimately create and test the .deb file.

>
> On 27.02.2015 19:09, Andres Riancho wrote:
>>
>> Guys,
>>
>>      Just found a github ticket you might find interesting.
>>      https://github.com/andresriancho/w3af/issues/3351
>>
>> On Thu, Feb 26, 2015 at 10:37 AM, Nich Ramsey <oni...@gm...> wrote:
>>>
>>> Hi Andres,
>>>
>>> I just started building the packages with virtualenv on a local vm.
>>> On Feb 26, 2015 2:48 AM, "Andres Riancho" <and...@gm...>
>>> wrote:
>>>>
>>>>
>>>> Sergey,
>>>>
>>>> On Thu, Feb 26, 2015 at 1:30 AM, Sergey <w3...@ko...> wrote:
>>>>>
>>>>> Hi, Andres and everybody.
>>>>>
>>>>> Right now I see that we have working CI builds of w3af Docker images.
>>>>
>>>>
>>>> We do! I've been working on the docker images last week and you can
>>>> see the latest in the develop branch :)
>>>>
>>>>> I'd like to know if anybody has some setup for building of w3af debian
>>>>> packages. For example using virtualenv/dh-virtualenv or fabric/robe or
>>>>> something like this?
>>>>
>>>>
>>>> Not that I know of, but you might be interested in this email thread
>>>> [0] where we discuss building kali packages in an automated way. The
>>>> summary is:
>>>>   * I would love to have automated builds of .deb
>>>>   * We could use docker images for testing the created deb packages in
>>>> Debian/Kali/etc.
>>>>   * I've been using circleci.com and would like to continue using that
>>>> CI system (free for open source)
>>>>   * This repository is the closest thing we have to an automated .deb
>>>> package [1] build
>>>>
>>>> If you want to help, let me know and we can draft a plan.
>>>>
>>>> [0]
>>>>
>>>> http://sourceforge.net/p/w3af/mailman/w3af-develop/thread/CA%2B1Rt66cek7ubXJHYe%2BbYxbUZg1HyvRDH7DViQkbUbvbWCxPLA%40mail.gmail.com/
>>>> [1] https://github.com/andresriancho/w3af-kali/
>>>>
>>> I haven't worked with circleci before, but I would be willing to go along
>>> with Sergey in this if he wants to. If he wanted to tackle the circleci
>>> alone, I could just share what I learn from the virtualenv build process.
>>>
>>>
>>>>> Thank you.
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Dive into the World of Parallel Programming The Go Parallel Website,
>>>>> sponsored
>>>>> by Intel and developed in partnership with Slashdot Media, is your hub
>>>>> for all
>>>>> things parallel software development, from weekly thought leadership
>>>>> blogs to
>>>>> news, videos, case studies, tutorials and more. Take a look and join
>>>>> the
>>>>> conversation now. http://goparallel.sourceforge.net/
>>>>> _______________________________________________
>>>>> W3af-develop mailing list
>>>>> W3a...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Andrés Riancho
>>>> Project Leader at w3af - http://w3af.org/
>>>> Web Application Attack and Audit Framework
>>>> Twitter: @w3af
>>>> GPG: 0x93C344F3
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Dive into the World of Parallel Programming The Go Parallel Website,
>>>> sponsored
>>>> by Intel and developed in partnership with Slashdot Media, is your hub
>>>> for
>>>> all
>>>> things parallel software development, from weekly thought leadership
>>>> blogs
>>>> to
>>>> news, videos, case studies, tutorials and more. Take a look and join the
>>>> conversation now. http://goparallel.sourceforge.net/
>>>> _______________________________________________
>>>> W3af-develop mailing list
>>>> W3a...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>>
>>
>>
>>
>



-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

Re: [W3af-develop] Continuous integration to debian packages

[Sergey <w3...@ko...>]

Thank you, Andres.

I'm not sure that w3af-kali fits my needs but it's definitely worth to 
look at it.

As for the ticket, how much it would take to resolve it? What are the 
main problems?

And btw how are dependencies' for w3af resolved in kali? Looks like 
package may become broken by some unanticipated update of w3af requirements.

On 27.02.2015 19:09, Andres Riancho wrote:
> Guys,
>
>      Just found a github ticket you might find interesting.
>      https://github.com/andresriancho/w3af/issues/3351
>
> On Thu, Feb 26, 2015 at 10:37 AM, Nich Ramsey <oni...@gm...> wrote:
>> Hi Andres,
>>
>> I just started building the packages with virtualenv on a local vm.
>> On Feb 26, 2015 2:48 AM, "Andres Riancho" <and...@gm...> wrote:
>>>
>>> Sergey,
>>>
>>> On Thu, Feb 26, 2015 at 1:30 AM, Sergey <w3...@ko...> wrote:
>>>> Hi, Andres and everybody.
>>>>
>>>> Right now I see that we have working CI builds of w3af Docker images.
>>>
>>> We do! I've been working on the docker images last week and you can
>>> see the latest in the develop branch :)
>>>
>>>> I'd like to know if anybody has some setup for building of w3af debian
>>>> packages. For example using virtualenv/dh-virtualenv or fabric/robe or
>>>> something like this?
>>>
>>> Not that I know of, but you might be interested in this email thread
>>> [0] where we discuss building kali packages in an automated way. The
>>> summary is:
>>>   * I would love to have automated builds of .deb
>>>   * We could use docker images for testing the created deb packages in
>>> Debian/Kali/etc.
>>>   * I've been using circleci.com and would like to continue using that
>>> CI system (free for open source)
>>>   * This repository is the closest thing we have to an automated .deb
>>> package [1] build
>>>
>>> If you want to help, let me know and we can draft a plan.
>>>
>>> [0]
>>> http://sourceforge.net/p/w3af/mailman/w3af-develop/thread/CA%2B1Rt66cek7ubXJHYe%2BbYxbUZg1HyvRDH7DViQkbUbvbWCxPLA%40mail.gmail.com/
>>> [1] https://github.com/andresriancho/w3af-kali/
>>>
>> I haven't worked with circleci before, but I would be willing to go along
>> with Sergey in this if he wants to. If he wanted to tackle the circleci
>> alone, I could just share what I learn from the virtualenv build process.
>>
>>
>>>> Thank you.
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Dive into the World of Parallel Programming The Go Parallel Website,
>>>> sponsored
>>>> by Intel and developed in partnership with Slashdot Media, is your hub
>>>> for all
>>>> things parallel software development, from weekly thought leadership
>>>> blogs to
>>>> news, videos, case studies, tutorials and more. Take a look and join the
>>>> conversation now. http://goparallel.sourceforge.net/
>>>> _______________________________________________
>>>> W3af-develop mailing list
>>>> W3a...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>>>
>>>
>>>
>>> --
>>> Andrés Riancho
>>> Project Leader at w3af - http://w3af.org/
>>> Web Application Attack and Audit Framework
>>> Twitter: @w3af
>>> GPG: 0x93C344F3
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Dive into the World of Parallel Programming The Go Parallel Website,
>>> sponsored
>>> by Intel and developed in partnership with Slashdot Media, is your hub for
>>> all
>>> things parallel software development, from weekly thought leadership blogs
>>> to
>>> news, videos, case studies, tutorials and more. Take a look and join the
>>> conversation now. http://goparallel.sourceforge.net/
>>> _______________________________________________
>>> W3af-develop mailing list
>>> W3a...@li...
>>> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>
>
>