Dragonfly v1.31.0

This release is mostly focused on stability features and bug fixes.

Important stability fixes and changes:

1. Introduce partial sync feature that avoids full replica reload during master/replica short disconnects [#5182]
2. Add support for GAT/GATS commands [#5257]
3. Introduce experimental huffman encoding for keys [#5179]

What's Changed

• chore: support fetching a huffman encoded string from CompactObj by @romange in https://github.com/dragonflydb/dragonfly/pull/5086
• fix(hset_family): Fix val being overwritten by TTL by @abhijat in https://github.com/dragonflydb/dragonfly/pull/5094
• fix: memcached authentication handling by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5098
• fix: memcached port has been already used crash by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5097
• chore: Add suffix and infix search syntax support in query parser by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5091
• chore(server): cleanup unused noop opcode in journal by @adiholden in https://github.com/dragonflydb/dragonfly/pull/5101
• chore: add arch type to dfly bench release binary by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/5090
• feat: add HELP SCAN and MINMSZ filter by @romange in https://github.com/dragonflydb/dragonfly/pull/5099
• fix test: rdb test keep loading expired keys by @adiholden in https://github.com/dragonflydb/dragonfly/pull/5105
• refactor: provide CommandContext into InvokeCmd by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/5100
• dfly_bench: Send auth if required before cluster setup by @abhijat in https://github.com/dragonflydb/dragonfly/pull/5107
• chore: tls improvements by @romange in https://github.com/dragonflydb/dragonfly/pull/5116
• fix: XTRIM command crash by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5121
• chore: update module versions by @romange in https://github.com/dragonflydb/dragonfly/pull/5126
• chore: provide metrics for huffman encoding by @romange in https://github.com/dragonflydb/dragonfly/pull/5117
• fix: make search keywords case insensitive by @romange in https://github.com/dragonflydb/dragonfly/pull/5129
• chore: replica clean ups by @romange in https://github.com/dragonflydb/dragonfly/pull/5127
• fix: collapsing logic in CapturingReplyBuilder by @romange in https://github.com/dragonflydb/dragonfly/pull/5133
• fix: improved socket error logging for connection diagnostics by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5062
• fix: out of bounds SIMD access by @romange in https://github.com/dragonflydb/dragonfly/pull/5140
• chore: improve logs around the parser error by @romange in https://github.com/dragonflydb/dragonfly/pull/5144
• feat: introduce resp validator by @romange in https://github.com/dragonflydb/dragonfly/pull/5147
• chore: minor fixes in geo_family by @romange in https://github.com/dragonflydb/dragonfly/pull/5149
• refactor: geosearch cmd with CmdArgParser by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/5153
• feat: add Georadius command by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/5158
• pytest: Add sleep to let master stop propagate to replica by @abhijat in https://github.com/dragonflydb/dragonfly/pull/5157
• chore: enable simd optimizations for aarch64 by @romange in https://github.com/dragonflydb/dragonfly/pull/5150
• fix: prevent crash in GEORADIUS argument parsing by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5166
• chore: more tests around dash table by @romange in https://github.com/dragonflydb/dragonfly/pull/5168
• chore: Add option to ignore parsing errors in replay tool by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5167
• fix: Prevent crash in PFCOUNT when using keys of different types by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5161
• chore: lower log level for harmless "Invalid Json path" warnings by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5163
• chore: remove global current_reply_size by @romange in https://github.com/dragonflydb/dragonfly/pull/5057
• replication: Ensure snapshot fiber joined in all code paths by @abhijat in https://github.com/dragonflydb/dragonfly/pull/5171
• chore: refactorings in dash code. by @romange in https://github.com/dragonflydb/dragonfly/pull/5178
• chore(dash_table): pass memory resource to segment by @romange in https://github.com/dragonflydb/dragonfly/pull/5176
• fix: show proactor kind on test failures by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/5170
• fix(parser): Parse correctly split simple response or error msg in client mode by @mkaruza in https://github.com/dragonflydb/dragonfly/pull/5177
• chore: introduce azure snapshot storage skeleton by @romange in https://github.com/dragonflydb/dragonfly/pull/5104
• fix: improve parsing of client facing responses by @romange in https://github.com/dragonflydb/dragonfly/pull/5181
• server(fix) : SCAN command reduce latency by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5151
• chore(json_family): Clean up the memory tracking code for JSON mutate operations. FOURTH PR by @BagritsevichStepan in https://github.com/dragonflydb/dragonfly/pull/5070
• feat(search_family): Ignore unsupported options in the FT.CREATE command by @BagritsevichStepan in https://github.com/dragonflydb/dragonfly/pull/5188
• fix (server): fix replication partial sync flow by @adiholden in https://github.com/dragonflydb/dragonfly/pull/5189
• feat: add flags support for huffman encoding by @romange in https://github.com/dragonflydb/dragonfly/pull/5179
• fix server: expire if need check valid condition fix by @adiholden in https://github.com/dragonflydb/dragonfly/pull/5193
• chore: minor dash fixes by @romange in https://github.com/dragonflydb/dragonfly/pull/5191
• fix(json_family): Fix memory tracking for the JSON.DEL command. FIFTH PR by @BagritsevichStepan in https://github.com/dragonflydb/dragonfly/pull/5075
• fix: prevent crash on KEYLOCK_ACQUIRED check for NO_KEY_TRANSACTIONAL commands by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5185
• fix(cluster_family): Cancel slot migration from incoming node on OOM by @mkaruza in https://github.com/dragonflydb/dragonfly/pull/5000
• fix: timeout for test_seeder_key_target by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/5198
• fix: crash in HLL PFMERGE with corrupted data by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5201
• fix: ZCOUNT crash when min > max by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5205
• fix: SearchStats race condition crash during concurrent FT operations by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5207
• fix(pytest): Sleep in sporadically failing test_slot_migration_oom pytest by @mkaruza in https://github.com/dragonflydb/dragonfly/pull/5214
• fix(transaction): DCHECK fail in non-atomic transactions by @romange in https://github.com/dragonflydb/dragonfly/pull/5217
• fix: seeder hset commands generation by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/5220
• fix(http_api): Return valid JSON string by @mkaruza in https://github.com/dragonflydb/dragonfly/pull/5210
• feat: add Lua GC flags by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/5194
• fix: Unsupported status for Copy/Move/Rename operations by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5223
• (string_family): Add priliminary support for GAT by @abhijat in https://github.com/dragonflydb/dragonfly/pull/5229
• chore: reduce leaky interface of DashCursor by @romange in https://github.com/dragonflydb/dragonfly/pull/5230
• chore: minor changes in dash code by @romange in https://github.com/dragonflydb/dragonfly/pull/5224
• test: add migration_restart_test with another slots set by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/5231
• chore: bring back partial sync by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/5182
• fix: macos build by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5234
• chore: maintain meta information around continous transactions by @romange in https://github.com/dragonflydb/dragonfly/pull/5232
• fix: dcheck trigger in JournalFlushGuard constructor by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/5241
• fix: typo in AclFamily by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5244
• fix(pubsub): Skip stale pubsub messages by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/5246
• chore: improve partial sync and test by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/5195
• chore(search): Add self time and resp3 format to ft.profile by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/5250
• fix: test_pubsub_unsubscribe test timeout increased by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5253
• chore: add corruption check for lpSetTotalBytes by @romange in https://github.com/dragonflydb/dragonfly/pull/5252
• chore: Pull latest helio by @mkaruza in https://github.com/dragonflydb/dragonfly/pull/5254
• fix: BITFIELD: overflow handling, type validation, and return values by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5242
• fix: prevent writing zero size fields into log file by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/5256
• fix(transaction): Fix non-transactional multi/exec transaction access by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/5260
• fix(search_family): Fix empty key bug by @BagritsevichStepan in https://github.com/dragonflydb/dragonfly/pull/5264
• chore: remove old ZIPLIST encoding during snapshotting by @romange in https://github.com/dragonflydb/dragonfly/pull/5267
• chore: stats for lsn buffer by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/5243
• chore: minor fixes in dash table by @romange in https://github.com/dragonflydb/dragonfly/pull/5274
• fix(json): Ensure full deletion for multi-match JSONPaths by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5270
• fix(reply_builder): Flush if last message didn't reply and flush but supose to by @mkaruza in https://github.com/dragonflydb/dragonfly/pull/5269
• fix(workflow): upload dev release to aws s3 also by @Abhra303 in https://github.com/dragonflydb/dragonfly/pull/5262
• chore: move MemoryBytesFlag to facade by @romange in https://github.com/dragonflydb/dragonfly/pull/5281
• dfly_bench: Support terminating early by @abhijat in https://github.com/dragonflydb/dragonfly/pull/5285
• refactor: remove is_oom flag from ConnectionContext by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/5216
• fix(protocol_client): Check if sock_ is not null before calling class method by @mkaruza in https://github.com/dragonflydb/dragonfly/pull/5291
• chore: add -skip-time-min flag to skip initial N minutes during replay traffic by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5288
• fix(json_family): Fix bugs where JSON commands were modifying values of other data types by @BagritsevichStepan in https://github.com/dragonflydb/dragonfly/pull/5266
• fix: Unsupported status 4 (XCLAIM command) by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5300
• Revert "fix(stream_family): Fix stream memory tracking issues (#5024)" by @romange in https://github.com/dragonflydb/dragonfly/pull/5301
• chore(dash): pass segment id to segment by @romange in https://github.com/dragonflydb/dragonfly/pull/5298
• Revert "fix(stream_family): Fix memory tracking for the STREAMS (#4781)" by @romange in https://github.com/dragonflydb/dragonfly/pull/5304
• feat: Added store option for sort command. by @H4R5H1T-007 in https://github.com/dragonflydb/dragonfly/pull/5095
• fix: prevent rename cmd crash by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/5307
• fix(string_family): Add RemoveDoc for the SET command by @BagritsevichStepan in https://github.com/dragonflydb/dragonfly/pull/5306
• chore: minor refactoring around AutoUpdater by @romange in https://github.com/dragonflydb/dragonfly/pull/5310
• fix(transaction): Fix schedule queues by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/4925
• chore(tiering): Test utils for faulty io by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/4928
• fix: huge sbf deserialization by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/5313
• fix: partial replication data loss by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/5297
• chore(string_family): Refactor to prep for adding the GAT command by @abhijat in https://github.com/dragonflydb/dragonfly/pull/5289
• chore: add another benchmark test for the GlobMatcher by @romange in https://github.com/dragonflydb/dragonfly/pull/5317
• fix: memory tracking bug in rename and SET by @romange in https://github.com/dragonflydb/dragonfly/pull/5315
• chore: Improve scope/aggregator usage in replies by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/5286
• chore(db_slice): Add object type to AddOrFind by @BagritsevichStepan in https://github.com/dragonflydb/dragonfly/pull/5305
• fix: fix db_capacity metrics description by @romange in https://github.com/dragonflydb/dragonfly/pull/5318
• Implementation of memcache GAT and GATS commands by @abhijat in https://github.com/dragonflydb/dragonfly/pull/5257
• fix: client kill preempts in atomic section on shutdown by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/5283
• fix(json): JSON.DEL recursive descent duplicate counting for nested same-key elements by @vyavdoshenko in https://github.com/dragonflydb/dragonfly/pull/5312
• chore(search_family): Add AddGenericDoc/RemoveGenericDoc methods by @BagritsevichStepan in https://github.com/dragonflydb/dragonfly/pull/5320
• chore: integrate Valkey fixes into stringmatchlen_impl by @romange in https://github.com/dragonflydb/dragonfly/pull/5323
• docs(readme): add translated documentation for pt-br by @Santosl2 in https://github.com/dragonflydb/dragonfly/pull/5276
• feat(json_family): Copy json_test.py::test_reset_key_to_string to json_family_test by @BagritsevichStepan in https://github.com/dragonflydb/dragonfly/pull/5324
• fix: Handle bonus key + init error in squashing by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/5303
• feat (server): add on move hook to dash insert new and bumpup by @adiholden in https://github.com/dragonflydb/dragonfly/pull/5278

New Contributors

• @Santosl2 made their first contribution in https://github.com/dragonflydb/dragonfly/pull/5276

Full Changelog: https://github.com/dragonflydb/dragonfly/compare/v1.30.0...v1.31.0