Crawl websites, sync to vector databases, and power RAG applications. Pre-built integrations for LLM pipelines and AI assistants.
Build data pipelines that feed your AI models and agents without managing infrastructure. Crawl any website, transform content, and push directly to your preferred vector store. Use 10,000+ tools for RAG applications, AI assistants, and real-time knowledge bases. Monitor site changes, trigger workflows on new data, and keep your AIs fed with fresh, structured information. Cloud-native, API-first, and free to start until you need to scale.
Try for free
Atera all-in-one platform IT management software with AI agents
Ideal for internal IT departments or managed service providers (MSPs)
Atera’s AI agents don’t just assist, they act. From detection to resolution, they handle incidents and requests instantly, taking your IT management from automated to autonomous.
ddclient is a long-standing, Perl-based Dynamic DNS client that keeps DNS records updated when your public IP changes. It supports a wide roster of DNS providers and APIs, allowing both A and AAAA updates so IPv4 and IPv6 hosts remain reachable. The client can discover your current address via network interfaces or external check services, which is crucial for NAT’d home connections. It runs as a daemon or from cron, with configurable intervals, SSL/TLS for provider communication, and robust...
...It reads from a list of domains formatted as:
domain_name <tab> or ; optional_comment
and generates a detailed, Excel-compatible CSV report with columns including:
DOMAIN: Domain name
REG: Registrar
SOA, NS, MX, TXT, SPF, DMARC, MS, A, PTR: Common DNS records for comprehensive domain analysis
NOTE: Optional comments from the original input file
Whether you're managing multiple domains or auditing for cross-platform DNS consistency, DNS Crawler simplifies the process, offering a clear, structured output for easy review and reporting.
PHP VerifyMail is a developing class designed to prevent automatic registrations based on entering regexp-valid but not existent mail domain. It uses DNS and SYN to check mx server existance. Based on Net_DNS from PEAR, used as a basic "includes".
A simple DNS analysis tool providing both IPv4 & IPv6 queries for A, NS, CNAME, SOA, MB, MG, MR, NULL, WKS, PTR, HINFO, MINFO, MX, TXT, AAAA, DS, RRSIG, NSEC, DNSKEY records. Also provides authorithy check and DNSSEC support.
C# DNS Client Library (100% managed code). Query domains to get MX records, CNAME and Address resource records from your application. All common DNS query types supported. Supports user-defined servers.
perl-dyndns is project to provide cross platform (Win, Mac, Linux) solution for DHCP ISP-connected users, that can obtain permanent DNS, MX and webhosting service for http://www.dyndns.org/. PROJECT MOVED. See Github user jaalto.
A DNS resolver library written in pure PHP script. This library offers almost same functionalities of getmxrr() and checkdnsrr(). You may find this library helpful when you would like to retrieve MX(Mail exchange) records in win32 PHP environment.