v1.21.0

Dragonfly v1.21.0

Some prominent changes include:

• Alpha release of SSD Data tiering - enabled with flag --prefix some/path/basename
• Very basic support of multi-tenancy @3260
• HSETEX now supports NX option, see our docs.
• Added support for JSON.MERGE and for JSON.MSET.
• valkey's replica-announce-ip and --cluster--announc--ip are consolidated via --announce-ip flag.

What's Changed

• feat(server): master stop sending exec opcode to replica by @adiholden in #3289
• chore: optimize zpopminmax operations by @romange in #3291
• fix(json_family): Fix error in JsonFamilyTest.MGet by @BagritsevichStepan in #3285
• fix: define macro WITH_AWS in cmake when flag is ON by @kostasrim in #3296
• chore: Add 'memory arena show' command by @romange in #3298
• fix: missing logs on pytest failures #3255 by @BorysTheDev in #3272
• chore: refactor compact_object and introduce materialize method by @romange in #3300
• fix(acl): loading interleaved plain and hashed passwords by @kostasrim in #3297
• chore: Add CompactObj Raw methods by @romange in #3303
• chore: On invalid TTL, print the TTL first by @chakaz in #3306
• chore: moving functions + renaming in tiered storage by @romange in #3304
• fix(tests): Add missing awaits by @chakaz in #3273
• chore: refactoring around tiered storage by @romange in #3299
• feat: yield when serialization is in progress by @kostasrim in #3220
• refactor: Use DbContext, OpArgs and Transaction to access DbSlice by @chakaz in #3311
• chore: Separate tiered serialization format from object values by @romange in #3310
• chore: add more community links to README by @Niennienzz in #3308
• fix(cluster): Join on specified attempt id by @chakaz in #3305
• chore(acl): add test with requirepass and aclfile by @kostasrim in #3312
• chore: skip test_cluster_flushall_during_migration by @kostasrim in #3316
• fix (pytest): generate unique random dbfilename for tests by @adiholden in #3317
• chore: bypass decoding/encoding of the data when performing offloading by @romange in #3315
• refactor: acl helpers and global tables by @kostasrim in #3309
• chore(tiering): Fixes by @dranikpg in #3225
• fix(migration): Use transactions! by @dranikpg in #3266
• fix: forbid DFLYCLUSTER commads set for emulated cluster mode by @BorysTheDev in #3307
• feat(namespaces): Initial support for multi-tenant by @chakaz in #3260
• chore: improve dfly_bench stats by @romange in #3320
• chore: remove replace_deleted flag from hnswlib by @dranikpg in #3323
• feat(test): Improve benchmark workflow by @adiholden in #3330
• fix: Proper shutdown sequence with Namespaces by @chakaz in #3333
• fix(test): copy logs for failed test during TEARDOWN phase by @BorysTheDev in #3331
• fix(json_family): fix JSON.STRAPPEND command for JSON legacy mode by @BagritsevichStepan in #3264
• chore(tiering): add protection against overruning memory budget by @romange in #3327
• chore: Add coordinated omission mode by @romange in #3332
• chore: implement sequential pass without the overlapping traffic by @romange in #3335
• feat: add an option to flush serialized entries on threshold limit by @kostasrim in #3241
• feat(hset_family): Add NX option to HSETEX by @BagritsevichStepan in #3295
• Fix blocking commands moved error by @BorysTheDev in #3334
• fix: ub in RegisterOnChange and regression tests for big values by @kostasrim in #3336
• fix: Cancel outgoing migration when retrying / closing by @chakaz in #3339
• chore: Make KeyIndex iterable by @dranikpg in #3326
• fix: AllocationTracker::Remove return value was reversed by @romange in #3341
• chore: remove redundant metrics from memory stats by @romange in #3345
• fix: corruption in replication stream by @romange in #3344
• chore: clean up TaskQueue since we do not need multiple fibers for it by @romange in #3348
• chore: pull helio by @romange in #3350
• chore: Log connection context when issuing dangerous cmds by @chakaz in #3352
• chore: small rename and add dcheck on LocalBlockingCounter by @kostasrim in #3356
• refactor: reduce number of logs for cluster by @BorysTheDev in #3357
• chore: fixes to dfly_bench by @romange in #3353
• chore: fix test_parser_memory_stats flakiness by @romange in #3354
• chore(server): Introduce StringSetWrapper by @dranikpg in #3347
• fix: do not upload offload values on a first hit by @romange in #3360
• chore(tiering): Range functions + small refactoring by @dranikpg in #3207
• fix: failure in test_cluster_fuzzymigration by @romange in #3363
• fix(server): Require >=1 args to GETEX by @chakaz in #3366
• fix(transaction): Fix namespace access by @dranikpg in #3364
• chore: disable compression on big values by @kostasrim in #3358
• fix: protect OnJournalEntry with ConditionGuard by @kostasrim in #3367
• chore: Introduce CoolQueue by @romange in #3365
• chore: small fixes around tiering by @romange in #3368
• chore: add a test for HeapSize() function by @romange in #3349
• chore: simplify computation of used_mem_current by @romange in #3372
• chore: disable cluster_fuzzymigration by @kostasrim in #3373
• chore(replica): remove unused methods in the Replica class by @BagritsevichStepan in #3374
• fix(transaction): Properly store block cancel status by @dranikpg in #3371
• chore: Track db_slice table memory instantly by @romange in #3375
• chore: add mem test for big values and default the flag by @kostasrim in #3369
• update: replication_acks_interval flag to 1000 by @kostasrim in #3378
• chore: dfly_bench - print ongoing error counts by @romange in #3382
• chore: introduce a cool queue that gradually retires cool items by @romange in #3377
• chore: update cached stats inside PollExecution by @romange in #3376
• fix: Fix test_take_over_seeder by @chakaz in #3385
• chore: reenable evictions upon insertion to avoid OOM rejections by @romange in #3387
• fix: remove fiber guard from non atomic section by @kostasrim in #3381
• chore: do not preempt on db_slice::RegisterOnChange by @kostasrim in #3388
• fix: disable inline transactions when db_slice has registered callbacks by @kostasrim in #3391
• fix: test_big_value_serialization_memory_limit shutdown timeout by @kostasrim in #3390
• chore: set serialization_max_chunk_size to 1 byte by @kostasrim in #3379
• chore: tiered fixes by @romange in #3393
• fix(acl): remove none from acl categories by @kostasrim in #3392
• chore: tiering - make Modify work with cool storage by @romange in #3395
• fix: Fix unsupported object type rejson-rl in RedisInsight by @BagritsevichStepan in #3384
• Revert "chore: set serialization_max_chunk_size to 1 byte (#3379)" by @chakaz in #3398
• chore(tiering): Move cool entry warmup to DbSlice by @dranikpg in #3397
• fix: reenable macos builds by @romange in #3399
• chore: Don't print password to log on replica AUTH failure by @chakaz in #3403
• chore: Support setting the value of replica-priority by @chakaz in #3400
• feat: stabilize non-coordinated omission mode by @romange in #3407
• chore: cancel slot migrations on shutdown by @dranikpg in #3405
• chore: add db_slice lock to protect segments from preemptions by @kostasrim in #3406
• chore: increase timeout of regression tests by @kostasrim in #3412
• fix: crash with NS in multi/exec #3410 by @BorysTheDev in #3415
• fix: json.merge exception crash by @kostasrim in #3409
• fix(connection): Count memchached pipelined commands by @dranikpg in #3413
• chore: remove verbose printing of tests by @romange in #3420
• chore: Tiered fixes by @romange in #3401
• feat: Support non-root paths for json.merge by @romange in #3419
• chore: skip cluster tests if redis-server wasn't found by @BorysTheDev in #3416
• chore: expose metric that shows how many task submitters are blocked by @romange in #3427
• chore: optimize SendStringArrInternal even more by @romange in #3425
• fix(server): Implement SCRIPT GC command by @dranikpg in #3431
• chore: retire TEST_EnableHeartBeat by @romange in #3435
• feat(server): Support replica-announce-ip/port by @chakaz in #3421
• test(cluster): Migration replication test by @dranikpg in #3417
• chore: improve replication locks by @romange in #3436
• chore: reorganize EngineShard::Heartbeat by @romange in #3437
• chore: fix memcached pipeline test by @dranikpg in #3438
• chore: simplify master replication cancelation interface by @romange in #3439
• chore: reset serialization_max_chunk_size to 0 by @kostasrim in #3432
• feat: DEBUG REPLICA PAUSE now pauses fullsync by @romange in #3441
• test: fix test_disconnect_replica by @BorysTheDev in #3442
• fix: cluster_mgr.py to use CLUSTER MYID by @chakaz in #3444
• chore: disable serialization_max_chunk_size in regtests by @romange in #3445
• fix: properly seriailize meta buffer in SendStringArrInternal by @romange in #3455

Full Changelog: v1.20.0...v1.21.0