Search Results for "crawling"

GoSpider - Fast web spider written in Go. Fast web crawling. Brute force and parse sitemap.xml. Parse robots.txt. Generate and verify link from JavaScript files. Link Finder. Find AWS-S3 from response source. Find subdomains from the response source. Get URLs from Wayback Machine, Common Crawl, Virus Total, Alien Vault. Format output easy to Grep. Support Burp input. Crawl multiple sites in parallel.

...The project doesn’t just include simple XSS forms; it spans variants such as DOM-based issues, context-sensitive sinks, template mishandling, CSRF, open redirects, and mixed content problems. Each scenario is crafted to reflect how bugs appear in production—behind frameworks, in odd encodings, or across redirects—so scanners must demonstrate accurate crawling and context understanding. Because the behaviors are stable and documented, teams can run comparative tests over time and quantify regression or improvement in their pipelines. It’s equally useful for human training, giving analysts a safe playground to practice exploitation and triage skills.

...The WAFEP application serves as both the "attacker" website and the "target" website, and thus, should ideally be used in twin instances - one BEHIND the WAF (the defender/target website), and another before the WAF (the attacker website). The payloads can be executed manually through the WAFEP attacker website instance by activating one test case at a time, or automatically, by using a crawling mechanism such as the one implemented in ZAP, Burpsuite, etc. *Note* The target website should be configured in the attacker website FIRST, by accessing: /wafep/config/change-target.jsp

This module provides attack surface reduction enhancements against the HTTP Flood Attacks at the web application level. Massive crawling/scanning tools, HTTP Flood tools can be detected and blocked by this module via htaccess, firewall or iptables, etc. (like mod_evasive) You can use this module by including "iosec.php" to any PHP file which wants to be protected. You can test module here: http://www.iosec.org/test.php (demo) Watch the Proof of Concept video: http://goo.gl/dSiAL Hakin9 IT Security Magazine Article about IOSEC http://goo.gl/aQM4Di (different format -> http://goo.gl/JKMUPN) IJNSA Article at http://goo.gl/LLxRdX WP Plugin Page http://goo.gl/nF5nD CHANGES v.1.8.2 - Iptables Auto Ban Bash Script Included - Token Access via Implicit Deny - Reverse Proxy Support - reCAPTCHA Support Do you want more features? ...

...The crawler has been completely rewritten in v1.0 bringing a lot of improvements: improved the data visualization, interactive option to download files, increased speed in crawling, exports list of found files into a separated file (useful to crawl a site once, then download files and analyse them with FOCA), generate an output log in Common Log Format (CLF), manage basic authentication and more! Many of the old features has been reimplemented and the most interesting one is the capability of the crawler to search for directory indexing.

Safe3WVS is one of the most powerful web vulnerability scanner with AI on-the-fly web spider crawling technology,especially web portals ,it is the most fast tool to dig such as sql injection, upload vulnerability, and more.http://www.safe3.com.cn/en