Stoolap Files

What's New in v0.2.0

A major release focused on performance, new features, and critical bug fixes.

New Features

• GROUPING SETS - Advanced aggregation with ROLLUP, CUBE, GROUPING SETS
• Volcano-style operator architecture - Industry-standard iterator model for joins (HashJoin, MergeJoin, NestedLoop, IndexNestedLoop)
• SEMI/ANTI join operators - Native operators for EXISTS/NOT EXISTS optimization
• Query timeout support - execute_with_timeout() and query_with_timeout() API
• Keyset pagination - O(limit) pagination for WHERE id > X ORDER BY id LIMIT Y

Critical Bug Fixes

• Hash collision bugs - Fixed data corruption in DISTINCT, GROUP BY, UNION, INTERSECT, EXCEPT where hash collisions could merge/lose rows
• EXISTS visibility bug - Row visibility now properly verified instead of trusting index
• UTF-8 corruption - Fixed non-ASCII character corruption in query normalization
• NULL handling in COUNT(DISTINCT) - NULLs now correctly excluded per SQL standard

Performance Optimizations

Query Execution

• Inline COALESCE/CASE/CONCAT evaluation bypassing VM
• Streaming global aggregation for AVG(col) * 100
• COW-based VM execution with zero-copy value handling
• Expression caching with LRU cache (256 entries)
• Query classification cache to avoid AST re-traversal
• LIMIT pushdown through joins, UNION ALL, and subqueries

Storage & MVCC

• O(1) transaction cache with HEAD-first visibility checks
• Arc-based row storage for O(1) read clones
• Batch MVCC operations reducing lock contention
• Arena-based storage with sync_arena() for fast scans
• Lazy scanner with cursor-based pagination (256 row batches)

Joins

• Hybrid streaming/parallel execution based on cardinality
• BloomFilter operator for probe-side pre-filtering
• JoinHashTable with 75% load factor and full hash storage
• Multi-column bloom filter (fixed single-key bug)
• Arc<Row> probe rows for O(1) clone in 1:N joins
• Projection pushdown in index nested loop join

Memory & Allocations

• CompactString for Value::Text (inline ≤24 bytes)
• Arc<Schema> throughout storage (130-617x faster schema access)
• Cached lowercase column names in Schema
• FxHashMap/AHashMap migration for faster hashing
• SmallVec for VM stack (inline capacity 16)
• Thread-local caches for regex, subquery results, semi-join sets

Aggregation & Window Functions

• Streaming DISTINCT with early termination
• COUNT(DISTINCT col) pushdown via index
• Type-specific sort paths for ORDER BY (25-35% faster)
• Columnar window ORDER BY: O(K) vs O(N) allocations
• Parallel sorting for large partitions (>10K rows)

Index Optimizations

• Sorted row IDs in indexes for O(n+m) merge operations
• Index nested loop for correlated EXISTS subqueries
• IN subquery/list index optimization for PK columns
• Secondary index fast path for OR/AND expressions
• Batch IN lookup with single lock acquisition

Architecture Improvements

• Expression VM - Bytecode-style compiled evaluation with stack-based execution
• Operator trait - open/next/close pattern for streaming execution
• JoinExecutor - Algorithm selection and orchestration
• RowFilter/JoinFilter - Thread-safe expression evaluation
• Zero-copy optimizations - Arc throughout query pipeline

Code Quality

• Major executor restructuring into focused modules
• Replace unsafe Send+Sync with safe-by-construction types
• validate_arg_count macro for scalar functions
• Typed error variants replacing internal errors
• Apache license headers

Full Changelog

https://github.com/stoolap/stoolap/compare/v0.1.0...v0.2.0