Search Results for "search engine project on java"

theHarvester is a very simple to use, yet powerful and effective tool designed to be used in the early stages of a penetration test or red team engagement. Use it for open source intelligence (OSINT) gathering to help determine a company's external threat landscape on the internet. The tool gathers emails, names, subdomains, IPs and URLs using multiple public data sources.

Robin is an AI-powered open source tool designed to assist investigators and researchers in conducting dark web OSINT (Open Source Intelligence) investigations. It combines automated dark web search capabilities with large language models (LLMs) to analyze and summarize information discovered across hidden services and Tor-based search engines. The tool helps refine investigative queries, collect results from multiple dark web sources, and filter relevant intelligence using AI-driven...

...It also tracks IP ranges used by search engine bots and automated agents including Googlebot, Bingbot, and OpenAI’s GPTBot. Lists are published in both IPv4 and IPv6 formats and are regularly updated through automated processes to keep the data current. In addition to provider specific lists, the project also offers merged and combined datasets that aggregate ranges from multiple sources into a single file.