Search Results for "text search"

PageIndex is an innovative open-source framework that reimagines retrieval-augmented generation (RAG) by eliminating conventional vector similarity search and instead building hierarchical semantic indexes that mirror a document’s natural structure. Rather than chunking text and embedding it into a vector database, PageIndex constructs a tree-structured index — similar to a detailed, AI-enhanced table of contents — that a large language model can traverse to locate the most relevant sections of long documents. ...

...As a CLI, it lets you build databases from structured data in one line, run queries against local files or in-memory databases, output results as JSON, CSV, or pretty tables, and configure full-text search. As a library, it exposes high-level APIs for inserting records, creating or transforming tables, normalizing schemas, and running migrations that SQLite’s limited ALTER TABLE cannot handle directly. The project also embraces an ecosystem of plugins, so you can add custom SQL functions, extra commands, or UIs (including a terminal UI) via separate packages. ...

Fairseq(-py) is a sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language modeling and other text generation tasks. We provide reference implementations of various sequence modeling papers. Recent work by Microsoft and Google has shown that data parallel training can be made significantly more efficient by sharding the model parameters and optimizer state across data parallel workers. These ideas are encapsulated in the...