Crawl websites, sync to vector databases, and power RAG applications. Pre-built integrations for LLM pipelines and AI assistants.
Build data pipelines that feed your AI models and agents without managing infrastructure. Crawl any website, transform content, and push directly to your preferred vector store. Use 10,000+ tools for RAG applications, AI assistants, and real-time knowledge bases. Monitor site changes, trigger workflows on new data, and keep your AIs fed with fresh, structured information. Cloud-native, API-first, and free to start until you need to scale.
Try for free
Atera all-in-one platform IT management software with AI agents
Ideal for internal IT departments or managed service providers (MSPs)
Atera’s AI agents don’t just assist, they act. From detection to resolution, they handle incidents and requests instantly, taking your IT management from automated to autonomous.
This project aims to create a method able to determine the most frequent word phrases in a large source of text data (>5 Gb) using the computational power of multiple processors.
Distributed event bus library for passing application events to listeners running anywhere on the network. Library is a lightweight wrapper over the JGroups toolkit with event filtering features.
Sfeb is a Simple and Flexible Enterprise Bus. Its purpose is to share business components on a network, and make them accessible through various protocols(http, jms, ...). Sfeb can be adapted, to your needs with the Sfeb sdk.
Soapod provides an open-source, open-standards Java(TM) application server. It embraces and implements an integrated peer-to-peer, service-oriented architecture based on open standards such as the Web service stack and JSR standards.
A novel Grid System which is Python based and Cell powered. By extending Namespace into GridSpace, any objects are accesable throughout the Grid. And the codes are distributed executed and be JIT compiled into Cell SPE instructions automatically.
The OpenSSI project is a comprehensive clustering solution offering a full, highly available SSI environment for Linux. Goals for OpenSSI Clusters include availability, scalability and manageability, built from standard servers.
mpiGraph is designed to inspect the health and scalability of a high-performance interconnect while under heavy load. This is useful to detect hardware and software problems in a system, such as slow nodes, links, switches, or routing contention.
X-Trace is a tool that provides fine-grained visibility into large, complex distributed systems. It can be used by application developers as well as network operators to debug and develop distributed systems.
Total Network Visibility for Network Engineers and IT Managers
Network monitoring and troubleshooting is hard. TotalView makes it easy.
This means every device on your network, and every interface on every device is automatically analyzed for performance, errors, QoS, and configuration.
Simple Distributed Job Management Tools facilitates parallel, distributed execution of simple commands, on a network of UNIX-like machines. Components include a job dependency/exclusivity language and load-balanced remote execution facility.
Cognitive agent based social simulation toolkit (RBOT+MRS) / production system based on ACT-R (http://act-r.psy.cmu.edu) allows for modelling single actor cognitive experiments (RBOT) and multiple actors in a simulated (semiotic) world (RBOT + MRS).
CoDeK - Java Concurrent Development frameworK is a very simple, open-source, academic java library aimed to help the development of Java multi-threading concurrent applications.
Peruser is a open-source lightweight knowledge application framework. Peruser speaks XML and RDF and is implemented in Java. Peruser code runs within Cocoon, as a standalone webapp, as console programs invoked through ant, or as embeddable java code.
ClusterIt is a collection of clustering tools to allow simple administration of groups of UNIX machines. It can also be used to facilitate parallel shell scripting, and compiling.
Distributed Inter-Process Communication (DIPC) is a combination of a Linux kernel extension and user-space programs that allows the user to develop distributed applications very easily.
Semys consists of a library for group communication and an architecture to build a sequential consistent system. A sequential consistent process can run on several physical hosts and is not disturbed by the downtime (crash) of a host.
xmote is a standard for exchanging data in a compact standardized XML format. In addition to defining the standard, xmote aims to provide a fully compliant and easy to use reference implementation.
FedStage OpenDSP is an open implementation of SOAP Web Service multi-user access and policy-based job control using OGF DRMAA routines supported by distributed resource management systems like Sun Grid Engine (SGE), LSF, PBSPro, Torque or Condor.
This application helps balances incoming TCP connections. This was intended mainly for VNC, but it can be used to balance other types of TCP connections.
metahelper is a utility which makes creating and maintaining upgradeable, removable, and verifiable configuration "metapackages" easy. A metapackage works with files owned by other packages to customize and configure them specifically for your environmen
OSN is an open source open protocol distributed social network. Public key cryptography makes the network resilient to spam. User profiles are based on FOAF XML and users can migrate their profile from one site of the federation to another.