Search Results for "crawler"
Sort By:
Showing 8 open source projects for "crawler"
View related business solutions-
Go From AI Idea to AI App FastAccess Gemini 3 and 200+ models. Build chatbots, agents, or custom models with built-in monitoring and scaling.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
IPRanges
Daily updated lists of cloud, bot, and service IP ranges
ipranges is an open source repository that provides continuously updated lists of IP address ranges associated with major cloud providers, search engine crawlers, and online services. ipranges collects IP ranges from publicly available sources and organizes them into structured files that can be easily used in security, networking, and automation workflows. It includes address ranges from providers such as Google Cloud, Amazon AWS, Microsoft, Oracle Cloud, and DigitalOcean, as well as well... -
2
TorBot
Dark Web OSINT Tool
Contributions to this project are always welcome. To add a new feature fork the dev branch and give a pull request when your new feature is tested and complete. If its a new module, it should be put inside the modules directory. The branch name should be your new feature name in the format <Feature_featurename_version(optional)>. On Linux platforms, you can make an executable for TorBot by using the install.sh script. You will need to give the script the correct permissions using chmod +x... -
3
ReconSpider
Most Advanced Open Source Intelligence (OSINT) Framework
...Reconnaissance is a mission to obtain information by various detection methods, about the activities and resources of an enemy or potential enemy, or geographic characteristics of a particular area. A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering). -
4
Photon
Incredibly fast crawler designed for OSINT
Photon is an extremely fast web crawler built specifically for OSINT and reconnaissance use cases. It is designed to extract URLs, endpoints, files, and other intelligence artifacts from target websites with minimal overhead. The crawler prioritizes speed and breadth, making it suitable for mapping web attack surfaces and discovering hidden resources. Photon is commonly used during early reconnaissance phases to build a comprehensive inventory of reachable assets. -
Stop Cyber Threats with VM-Series Next-Gen Firewall on AzureGain integrated visibility across all traffic in a single pass. Deploy Palo Alto Networks VM-Series to determine application identity and content while automating security policy updates via rich APIs.
-
5
Domain Analyzer Security Tool
Finds all the security information for a given domain name
Domain analyzer is a security analysis tool which automatically discovers and reports information about the given domain. Its main purpose is to analyze domains in an unattended way. -
6
SauceWalk Proxy Helper
Enumeration and automation of file discovery for your sec tools.
...The remote agent can be used to identify target files and folders on a live system via a PHP script on the target server(ASP/JSP coming soon). The advantage of this tool is that it allows access to files and folders (for example include or plugin folders) which are not usually seen via a spider or crawler to be security tested with traditional tools. The Py version is on its way soon. -
7
Web Crawler Security Tool
A web crawler oriented to information security.
...Many of the old features has been reimplemented and the most interesting one is the capability of the crawler to search for directory indexing. -
8
Monkey-Spider
Moved to https://github.com/aikinci/monkeyspider
The Monkey-Spider is a crawler based low-interaction Honeyclient Project. It is not only restricted to this use but it is developed as such. The Monkey-Spider crawles Web sites to expose their threats to Web clients.
- Previous
- You're on page 1
- Next
