Stoolap Files

What's New in v0.4.1

v0.4.1 is a correctness and durability release on the v0.4.0 storage engine, with a long list of query performance work. It raises the minimum supported Rust version to 1.88.

Correctness

A differential campaign ran about 1,600 queries against SQLite 3.51 and DuckDB 1.2 and fixed every case where both engines agreed and Stoolap differed (#94, [#92], [#93], [#95], [#97]):

• Correlated subqueries read the parent row wherever they name it: the select list, an aggregate argument or FILTER, HAVING, CASE, ORDER BY, and through nested subqueries and derived tables. Correlated subqueries over views, derived tables and CTE results are evaluated per row.
• CTEs referenced more than once, in set-operation branches, the select list, HAVING, ORDER BY or a window, are materialized once and read from every place. A CTE's own ORDER BY expressions and DISTINCT are honoured.
• Joins: a residual predicate on an outer join no longer turns rejected pairs into preserved rows; the whole ON clause is applied where an index answers the key; cross-type numeric equi-join keys match (#50); a LIMIT join keeps fetching until the limit is met (#83); USING names its column by the bare name.
• Set operations: ORDER BY, LIMIT and OFFSET apply to the whole UNION / INTERSECT / EXCEPT, and OFFSET is applied once on every early-exit path (#86).
• Windows: named windows can be extended with OVER (w ...), a window the select list does not carry can sort, navigation functions read a qualified argument through its table (#93), same-named windows stay apart and NULLS FIRST / LAST is honoured (#88), NULL sort keys no longer tie with MAX values and a value of another type past the sampled rows no longer corrupts the order (#54), a constant argument no longer errors on an empty table (#53).
• Sorting: the radix sort orders NULLs correctly (#44), and ORDER BY + LIMIT keeps its rows through every early-exit path.
• Aggregates: COUNT(col) skips NULLs, booleans count as one or nought in SUM and AVG, an ORDER BY on a grouped column the SELECT drops is honoured (#59), and expressions that differ only by parentheses are distinct aggregates.
• Expressions: NOT over AND and OR follows three-valued logic in storage filters, a comparison against NULL stays unknown, x * 0 is not folded, simple CASE with a literal operand evaluates correctly, IN over non-constant items compiles and judges its value once, volatile functions such as RANDOM() are evaluated once per row, float constants against INTEGER columns compare numerically (#40).
• DML and DDL: DELETE removes what it matched on a table without a key; ON UPDATE / ON DELETE RESTRICT and cascades are checked before anything changes; INSERT ... ON CONFLICT DO UPDATE takes a WHERE and reports its errors; WITH before UPDATE and DELETE; SET col = DEFAULT writes the column default; table-level and composite PRIMARY KEY are keys; a transaction can reinsert a key it deleted; index-driven scans, updates and deletes see the rows the same transaction changed (#79).
• Storage: rows sealed into cold volumes stay visible to executor index probes (#82); two panics on valid SQL and the remaining i64::MIN and unbounded-LIMIT panics are gone (#56, [#57]).
• Caches: the semantic cache no longer serves one string IN list's result for another or answers col >= v from a cached col > v, the merge join and COUNT(col) answer right (#31), the compiled-program caches are keyed by two independent hashes so a collision cannot alias two expressions (#36), and the subquery result cache is no longer poisoned by a rollback (#49).

Durability

• WAL fsync happens at commit and DDL boundaries instead of per entry, with a poisoned-WAL floor on fsync failure: a 1,000-row transaction in sync_mode=full went from 3,742 ms to 10 ms (#34).
• Swallowed durability errors in tombstone renames, PK cleanup and manifest loading now fail loud (#32), and a database whose cold volumes cannot be reloaded fails closed instead of opening on partial data (#35).

Performance

Fixed per-statement costs on the cached SELECT, prepared INSERT, UPDATE and DELETE paths (#39, [#41], [#42], [#43]); join reordering in place, top-K ORDER BY LIMIT, per-row clone and re-hash removal in the result pipeline (#44, [#45], [#46]); recursive CTE working sets shared across iterations (#47, [#48]); memoized IN sets (#49); leaner hash and index nested loop joins with projected rows built from the two inputs (#51, [#81]); grouped GROUP BY + LIMIT joins streamed through the right side's index (#87, [#89], [#90], [#91]); raw-entry grouping and one allocation per group in storage GROUP BY (#52, [#61], [#62]); O(n) PERCENT_RANK and CUME_DIST and shared partition sorting across an OVER clause (#53, [#55], [#66]); tokens sliced from the parser input (#67); the hash index's write-only map and the 10,000-slot arena reservation per table dropped (#63, [#64]); IN-list ids probed on the primary key even when the list pushed down as a range (#76, [#77], [#78]); a correlated subquery reaches the primary key when names clash (#71); a subquery two levels down that reads only the outermost row folds once instead of running per row.

API

• Transaction::execute_prepared_named and query_prepared_named run a prepared statement with :name parameters inside a transaction without reparsing; the C API gains stoolap_exec_named, stoolap_query_named, stoolap_tx_exec_named, stoolap_tx_query_named, stoolap_tx_stmt_exec_named and stoolap_tx_stmt_query_named, declared in include/stoolap.h (#98).
• TRIM, LTRIM and RTRIM take the characters to strip as a second argument.

Compatibility

• Minimum supported Rust version is 1.88, enforced in CI (#37).
• Dialect kept as before and documented: NULLs sort first on DESC and last on ASC (PostgreSQL rule); backslash is an escape in single-quoted literals; CAST('3.9' AS INTEGER) and float equality follow SQLite. A primary key UPDATE is an explicit error.
• Test profiles: cargo nextest run leaves out the wall-clock and stress tests; cargo nextest run -P full runs everything.

Known gaps

UPDATE ... FROM, GROUPS BETWEEN frames, the EXCLUDE frame clause, WITH inside a subquery, WITH RECURSIVE ... UNION without ALL, several named windows in one WINDOW clause, a correlated subquery inside JOIN ON, and SELECT t.* ... ORDER BY other.col are not yet supported.