Crawl websites, sync to vector databases, and power RAG applications. Pre-built integrations for LLM pipelines and AI assistants.
Build data pipelines that feed your AI models and agents without managing infrastructure. Crawl any website, transform content, and push directly to your preferred vector store. Use 10,000+ tools for RAG applications, AI assistants, and real-time knowledge bases. Monitor site changes, trigger workflows on new data, and keep your AIs fed with fresh, structured information. Cloud-native, API-first, and free to start until you need to scale.
Try for free
Turn more customers into advocates.
Fight skyrocketing paid media costs by turning your customers into a primary vehicle for acquisition, awareness, and activation with Extole.
The platform's advanced capabilities ensure companies get the most out of their referral programs. Leverage custom events, profiles, and attributes to enable dynamic, audience-specific referral experiences. Use first-party data to tailor customer segment messaging, rewards, and engagement strategies. Use our flexible APIs to build management capabilities and consumer experiences–headlessly or hybrid. We have all the tools you need to build scalable, secure, and high-performing referral programs.
Ruya is a Python-based breadth-first, level-, delayed, event-based-crawler for crawling English, Japanese websites. It is targeted solely towards developers who want crawling functionality in their projects using API, and crawl control.
Aracnis is a Java based framework for building distributed web spiders. These spiders can be used to accomplish a variety of tasks, for example, screen-scraping and link integrity checking.
Rise Vision is the #1 digital signage company, offering easy-to-use cloud digital signage software compatible with any player across multiple screens. Forget about static displays. Save time and boost sales with 500+ customizable content templates for your screens. If you ever need help, get free training and exceptionally fast support.
JLinkCheck is an Ant Task written in Java for checking links in websites. It is not just checking one single page, but crawling a whole site like a spider, generating a report in XML and (X)HTML. JReptator will be its succesor with many more features
Sperowider Website Archiving Suite is a set of Java applications, the primary purpose of which is to spider dynamic websites, and to create static distributable archives with a full text search index usable by an associated Java applet.
InSite is a Web site management tool written in perl. It checks link integrity and does some basic content monitoring of your site's files directly on the local disk, which gives it a huge speed advantage over similar tools.
A first-of-its-kind homeownership solution that puts YOU at the center of the coveted lifetime consumer relationship.
BoldTrail, the #1 rated real estate platform, is built to power your entire brokerage with next-generation technology your agents will use and love. Showcase your unique brand with customizable websites for your company, offices, and every agent. Maximize lead capture with a modern, portal-like consumer search experience and intelligent behavior tracking. Hyper-local area pages, home valuation pages and options for rich lifestyle data keep customers searching with your brokerage as the local experts. The most robust lead gen tools on the market help your brokerage, teams & agents effectively drive new business - no matter their budget. Empower your agents to generate free leads instantly with our simple to use landing pages & IDX squeeze pages. Drive more leads with higher quality and lower cost through in-house tools built within the platform. Diversify lead sources with our automated social media posting, integrated Google and Facebook advertising, custom text codes and more.
404SEF is a component for Mambo CMS (4.5.x right now, 4.6.x soon) to provide Human Readable URLs. Works with apache and IIS. Provides proper 404 status code for missing content, logs 404 errors, and user-defined custom redirection via special shortcuts
Automatic link management program. Has three functions: List links in database in html format, add links to database using browser and optionaly check for bad links (by cron job). This eliminates the need for the "Report bad link" on too many web sites
Like social boomarking, allows users to share their bookmarks online. Like wiki, anyone can freely edit links. Export / Import boomarks with your browser. Many other features: RSS and Atom feeds, URL check, popular categories, XLIink, ...
The main function of this script is to shorten long website-URL's -- converting long URL's into easy-to-remember, short ones. [htaccess, MOD_REWRITE, XHTML 1.0 strict, CSS 1, JS 1.2, PHP 5X, MySQL 4X]
A content management system which allows web developers to create and organize a collection of URLs (a.k.a. - a link farm) using a searchable labeling system.
Bugkilla is a set of java tools for the functional test of J2EE Web Applications.
Specification and execution of tests will be automated for web front end and business logic layer.
One goal is to integrate with existing frameworks and tools.
Allow visitors to submit their links to your website. Links submitted by visitors are only added to the database once the administrator has approved them. More ..... www.upoint.info/cgi
The "WebNavigationFramework" (WNF) is a Environment to create or build
Websites and/or Webappls it knows Sessions for SingleSignOn and functions
for expanding.Nav.-nodes are User-, Date- and/or Locationspecific and hierarchically.
SiteEngine is a Servlet based web subsystem purposed to simplify of managing of site content (but it is not a Content Management System, it is just a lib).
This will be a PHP based website system. It will have an xml database for all content (no need for database software.) Also it will have the ability to use a template system that can easily accomodate flash and WYSYWIG tpl.