Search Results for "indexing"
Sort By:
Showing 47 open source projects for "indexing"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
1
PixelRAG
The beginning of scalable pixel-native search
...This approach preserves layout, tables, charts, diagrams, infographics, and other visual structure that traditional HTML or text parsing can miss. The project includes tools for rendering, chunking, embedding, indexing, and serving visual search indexes. It also provides a hosted API with a prebuilt Wikipedia index, plus local pipelines for building indexes from custom documents. PixelRAG can be used with Claude through the pixelbrowse skill, giving agents the ability to inspect pages visually instead of relying only on raw markup. -
2
pgai
A suite of tools to develop RAG, semantic search, and other AI apps
pgai is a suite of PostgreSQL extensions developed by Timescale to empower developers in building AI applications directly within their databases. It integrates tools for vector storage, advanced indexing, and AI model interactions, facilitating the development of applications like semantic search and Retrieval-Augmented Generation (RAG) without leaving the SQL environment. -
3
Engram
A New Axis of Sparsity for Large Language Models
...It provides utilities to generate embeddings from text or other structured data, index them using efficient approximate nearest neighbor algorithms, and perform real-time similarity queries even on large corpora. Engineered with speed and memory efficiency in mind, Engram supports batched indexing, incremental updates, and custom distance metrics so developers can tailor search behaviors to their domain’s needs. In addition to raw similarity search, the project includes tools for clustering, ranking, and filtering results, enabling richer user experiences like “related content”, semantic auto-completion, and contextual filtering. -
4
Pathway AI Pipelines
Ready-to-run cloud templates for RAG
...It supports numerous connectors including local files, Google Drive, SharePoint, Kafka, PostgreSQL, and real-time APIs, making it suitable for enterprise data environments. The templates include built-in indexing, vector search, hybrid search, and caching capabilities that remove the need to assemble separate infrastructure components. Developers can run the applications locally or deploy them to cloud platforms using Docker with minimal setup. Overall, llm-app functions as a practical accelerator for teams building real-time, production-ready AI knowledge systems. -
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
5
OpenViking
Context database designed specifically for AI Agents
OpenViking is an open-source context database engineered for efficient indexing and retrieval of large amounts of unstructured or semi-structured context data used by AI applications. It’s primarily designed to serve as a high-performance, scalable backend for storing app context, embeddings, conversational histories, and other textual artifacts that need rapid lookup and semantic search, which makes it especially useful for systems like chatbots or memory-augmented agents. -
6
LightRAG
"LightRAG: Simple and Fast Retrieval-Augmented Generation"
LightRAG is a lightweight Retrieval-Augmented Generation (RAG) framework designed for efficient document retrieval and response generation. It is optimized for speed and lower resource consumption, making it ideal for real-time applications. -
7
Basic Memory
Persistent AI memory using local Markdown knowledge graphs
...Basic Memory creates a semantic knowledge graph by linking related ideas, making it easier to retrieve, expand, and connect information over time. With a local-first design, your data stays private and portable, while optional cloud sync enables cross-device access. It combines simplicity with powerful indexing and search, giving you a flexible way to build long-term memory for projects, research, and workflows. -
8
MCP Agent Mail
Asynchronous coordination layer for AI coding agents
...Advisory file reservations let an agent declare intended edits to files or patterns, reducing accidental overlap without enforcing rigid locks. Git stores human-auditable communication artifacts, while SQLite supports indexing, search, and operational queries. The HTTP-only FastMCP server works with clients such as Claude Code, Codex, Gemini CLI, and other MCP-compatible tools. Additional capabilities include acknowledgments, project directories, product-wide communication, build slots, deployment helpers, and integration with dependency-aware task tracking. -
9
VideoRAG
"VideoRAG: Chat with Your Videos
VideoRAG is a retrieval-augmented generation (RAG) framework tailored for video content that enables AI systems to answer questions, summarize, and reason over long videos by combining visual embeddings with contextual search. The system works by first breaking video into clips, extracting visual and audio-textual features, and indexing them into embeddings, then using an LLM with a retriever to pull relevant segments on demand. When a user query is received, VideoRAG locates semantically relevant moments in the video using the embedding index, retrieves associated clips or transcripts, and feeds them to a generative model to produce accurate, grounded answers or summaries. ... -
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
10
GraphRAG
A modular graph-based Retrieval-Augmented Generation (RAG) system
The GraphRAG project is a data pipeline and transformation suite that is designed to extract meaningful, structured data from unstructured text using the power of LLMs. -
11
LEANN
Local RAG engine for private multimodal knowledge search on devices
...LEANN introduces a storage-efficient approximate nearest neighbor index combined with on-the-fly embedding recomputation to avoid storing large embedding vectors. By recomputing embeddings during queries and using compact graph-based indexing structures, LEANN can maintain high search accuracy while minimizing disk usage. It aims to act as a unified personal knowledge layer that connects different types of data such as documents, code, images, and other local files into a searchable context for language models. -
12
clip-retrieval
Easily compute clip embeddings and build a clip retrieval system
...The system is optimized for performance and scalability, capable of processing tens or even hundreds of millions of embeddings using GPU acceleration. It includes components for inference, indexing, filtering, and serving results through APIs, making it a complete pipeline for building production-ready retrieval systems. The framework also supports querying by image, text, or embedding, enabling flexible use cases such as reverse image search or multimodal content discovery. Additionally, it provides a simple frontend interface and backend services that can be deployed to expose search functionality to users. -
13
Cognita
Open source RAG framework for building scalable modular AI apps
...It includes both a backend service and a frontend interface, enabling users to upload documents, experiment with configurations, and perform question-answering tasks interactively. Cognita supports incremental indexing, meaning it processes only new or updated data to reduce computational overhead and improve efficiency. -
14
MiniRAG
Making RAG Simpler with Small and Open-Sourced Language Models
MiniRAG is a lightweight retrieval-augmented generation tool designed to bring the benefits of RAG workflows to smaller datasets, edge environments, and constrained compute settings by simplifying embedding, indexing, and retrieval. It extracts text from documents, codes, or other structured inputs and converts them into embeddings using efficient models, then stores these vectors for fast nearest-neighbor search without requiring huge databases or separate vector servers. When a query is issued, MiniRAG retrieves the most relevant contexts and feeds them into a generative model to produce an answer that is grounded in the source material rather than hallucinated. ... -
15
UltraRAG
Less Code, Lower Barrier, Faster Deployment
UltraRAG 2.0 is a low-code, MCP-enabled RAG framework that aims to lower the barrier to building complex retrieval pipelines for research and production. It provides end-to-end recipes—from encoding and indexing corpora to deploying retrievers and LLMs—so users can reproduce baselines and iterate rapidly. The toolkit comes with built-in support for popular RAG datasets, large corpora, and canonical baselines, plus documentation that walks from “quick start” to debugging and case analysis. It encourages pipeline composition via configuration, enabling researchers to swap retrievers, rerankers, and generators without heavy refactoring. ... -
16
EverMemOS
Long-term memory OS for AI with structured recall and context awarenes
...Instead of treating each prompt independently, it builds evolving user profiles, tracks preferences, and connects related events into coherent narratives. Its architecture combines memory storage, indexing, and retrieval with agent-level reasoning, allowing AI systems to make informed decisions based on prior interactions. EverMemOS goes beyond simple retrieval by actively applying stored knowledge to current tasks, improving personalization and consistency. EverMemOS uses a multi-stage memory lifecycle to convert raw dialogue into structured semantic data, supporting long-horizon reasoning and adaptive behavior across sessions. -
17
Swirl
Swirl queries any number of data sources with APIs
Swirl queries any number of data sources with APIs and uses spaCy and NLTK to re-rank the unified results without extracting and indexing anything! Includes zero-code configs for Apache Solr, ChatGPT, Elastic Search, OpenSearch, PostgreSQL, Google BigQuery, RequestsGet, Google PSE, NLResearch.com, Miro & more! SWIRL adapts and distributes queries to anything with a search API - search engines, databases, noSQL engines, cloud/SaaS services etc - and uses AI (Large Language Models) to re-rank the unified results without extracting and indexing anything. ... -
18
Sage Chat
Chat with any codebase in under two minutes | Fully local
...Developers can ask natural language questions about a project, and the system responds with explanations supported by references to the relevant code, documentation, or external technical resources. The project aims to act as a contextual knowledge layer for software teams by combining language models with repository indexing and documentation retrieval. Sage can operate locally or connect to external AI services, depending on the configuration, providing flexibility for privacy-sensitive environments. -
19
Pixeltable
Data Infrastructure providing an approach to multimodal AI workloads
Pixeltable is an open-source Python data infrastructure framework designed to support the development of multimodal AI applications. The system provides a declarative interface for managing the entire lifecycle of AI data pipelines, including storage, transformation, indexing, retrieval, and orchestration of datasets. Unlike traditional architectures that require multiple tools such as databases, vector stores, and workflow orchestrators, Pixeltable unifies these functions within a table-based abstraction. Developers define data transformations and AI operations using computed columns on tables, allowing pipelines to evolve incrementally as new data or models are added. ... -
20
RAG API
ID-based RAG FastAPI: Integration with Langchain and PostgreSQL
rag_api is an open-source REST API for building Retrieval-Augmented Generation (RAG) systems using LLMs like GPT. It lets users index documents, search semantically, and retrieve relevant content for use in generative AI workflows. Designed for rapid prototyping, it is ideal for chatbot development, document assistants, and knowledge-based LLM apps. -
21
shuyuan
Reading book source
...The name suggests “academy” or “study hall,” and the tool aims to help users ingest, organize, and manage reading content — possibly offering features like text parsing, annotation, metadata generation, translation, or storage for later reference. The repository is set up to support document ingestion, indexing, and maybe some AI-aided summarization or lookup functions, which helps users convert large text corpora into a structured, searchable knowledge base. For learners, researchers, or avid readers, Shuyuan offers a way to bridge from plain text files or eBooks into a manageable, interactive resource — one where notes, references, and reading progress can be tracked. ... -
22
gensim
Topic Modelling for Humans
Gensim is a Python library for topic modeling, document indexing, and similarity retrieval with large corpora. The target audience is the natural language processing (NLP) and information retrieval (IR) community. -
23
Milvus Bootcamp
Dealing with all unstructured data, such as reverse image search
Milvus Bootcamp is a collection of tutorials, examples, and best practices for using Milvus, an open-source vector database designed for AI-powered similarity search and retrieval applications. -
24
All-in-RAG
Big Model Application Development Practice 1
...The repository provides a structured learning path that covers both theoretical foundations and practical implementation steps for RAG systems. It explains the full development pipeline required to create knowledge-aware AI assistants, including data preparation, document indexing, vector embedding generation, and retrieval strategies. The project also explores advanced topics such as hybrid retrieval methods, query optimization, and evaluation techniques for improving system accuracy. Alongside theoretical explanations, the repository includes hands-on exercises and example projects that demonstrate how to build production-ready RAG systems. ... -
25
DB-GPT
Revolutionizing Database Interactions with Private LLM Technology
DB-GPT is an experimental open-source project that uses localized GPT large models to interact with your data and environment. With this solution, you can be assured that there is no risk of data leakage, and your data is 100% private and secure.