Search Results for "token"

rag-search is a lightweight Retrieval-Augmented Generation API service designed to provide structured semantic search and answer generation through a simple FastAPI backend. The project integrates web search, vector embeddings, and reranking logic to retrieve relevant context before passing it to a language model for response generation. It is built to be easily deployable, requiring only environment configuration and dependency installation to run a functional RAG service. The system...

...Support Chinese word segmentation (concurrent word segmentation using the sego word segmentation package, speed 27MB/sec). Support to calculate the proximity distance of keywords in the text (token proximity). When a request to add a document to the index comes in, the main coroutine will send the text to be segmented to a word segmentation coroutine through a channel, and the coroutine will segment the text and send it to a word segmentation through another channel. Indexer coroutines. The indexer coroutine builds an inverted index from the search keyword to the document, and the inverted index table is stored in memory for quick recall. ...