Semlith Files

Removes three of the four limitations 0.1.0 shipped with, cuts peak indexing memory to a third, and changes the default embedding model.

Changed

• New default embedding model: granite-embedding-small-english-r2, int8. Replaces BGESmallENV15 for newly created stores. On a 6260-chunk benchmark of mixed Rust, Markdown and TypeScript it scored 16.00 code MRR@10 against BGE-small's 14.84, in a 52 MB download instead of 133 MB, at the same query latency. It is 384-dimensional, so the index geometry is unchanged.

Existing stores are not affected and are not migrated. A store keeps the model it was built with, because vectors from two models are not comparable. To move an existing store onto the new default, delete the store directory and re-index.

• Search is now hybrid. Every query consults SQLite FTS5 for literal terms alongside the vector index, and the two rankings are fused by reciprocal rank. There is no flag: dense-only search cannot reliably find an exact identifier, which is one of the most common things asked of a code corpus. Existing stores have their keyword index built on first open, which costs no re-embedding.

The score on a hit is now a fusion score rather than a cosine similarity. It orders results within one query and means nothing across queries.

• ONNX Runtime thread count is chosen rather than defaulted. On Apple silicon it is the performance-core count; elsewhere the full core count. ONNX Runtime synchronises threads per operator, so one thread on an efficiency core paces the whole batch — measured 16.5 chunks/sec at four threads against 13.9 at eight on a 4P+4E M1. Override with SEMLITH_EMBED_THREADS.

• Minimum supported Rust version is now 1.89, for std::fs::File::try_lock.

Added

• One writer per store. An index run holds an OS advisory lock on the store for its duration. A second run exits non-zero naming the process that holds it, instead of interleaving writes until the vector index and the database disagree. The kernel releases the lock if the process dies, so an interrupted run leaves nothing to clean up and no stale file to delete by hand.

• semlith models lists the new default alongside fastembed's built-in models.

Fixed

• A release whose git tag disagrees with Cargo.toml now fails the release build instead of publishing binaries labelled with a version they were not built from.

Performance

Measured on a 4P+4E Apple Silicon laptop with 8 GB of RAM, over corpora of mixed Rust, Markdown and TypeScript.

• Peak indexing memory is now roughly 600 MB and no longer grows with the corpus, against ~1.7 GB in 0.1.0. Across a 1.2k, 9.9k and 105k chunk corpus it varies by 7 percent, and the largest corpus uses the least. Two causes: embedding batches were being flushed once per file rather than once per batch, so a single large file could hold thousands of chunks in memory; and the batch size itself was three times larger than it needed to be.
• Indexing throughput is up from ~13 to ~23 chunks/sec, from sizing the ONNX Runtime thread pool to the machine's performance cores.
• Retrieval quality, measured end to end through the binary over a 6527-chunk corpus with 1809 self-labelled queries:

The model change alone would have regressed prose retrieval. Keyword fusion more than recovers it, which is why the two ship together.

Library API

semlith is a published crate, so these are breaking changes to its Rust API. The library API is unstable before 1.0 and changes with the minor version.

• DEFAULT_MODEL: EmbeddingModel is replaced by default_model() -> Model.
• Semlith::open takes Option<Model> rather than Option<EmbeddingModel>, and Semlith::model returns &Model. Model is an enum over fastembed's built-ins and the new default, which is not one of them.
• New modules: embed (model identity and loading) and lock (store locking).

Install

Download the archive for your platform, unpack it, and put semlith on your PATH. Linux builds need OpenBLAS at runtime (apt-get install libopenblas-dev or your distribution's equivalent). Intel macOS has no prebuilt binary: ONNX Runtime no longer publishes x86_64 macOS builds.

Verify a download against SHA256SUMS before running it.