This is a patch release

What's Changed

• fix(tiering): improvements around tiering stability #6040 #6086 #6067
• fix(server): replica rejects replicating from itself #6097

Full Changelog: v1.35.0...v1.35.1

Dragonfly v1.35.0

Important stability fixes and changes:

• Defragmentation covers list and json data types: #5790, #6023
• FT.Search can now search tags based on prefix and suffix globs: #5871, #5895
• Added support for FT.Search GEO index: #5854
• Optional Huffman compression for keys and values: #6025, #6037

What's Changed

• feat(core): support defragmentation for quicklist by @abhijat in #5790
• chore: skip arguments when logging failed auth cmd by @kostasrim in #5820
• fix(server): Fix AllocatingTracker crash due to VLOG by @dranikpg in #5827
• fix(server): Resest global state on loading error by @dranikpg in #5825
• core: use explicit types and minor cleanups by @abhijat in #5826
• server: Fix metric name by @abhijat in #5829
• fix(server): Support FLUSH(ALL) SYNC by @dranikpg in #5821
• fix(server): Use correct messages for sharded pubsub by @dranikpg in #5818
• fix: stack overflow in hincrbyfloat by @BorysTheDev in #5835
• fix: mac-os build by @romange in #5840
• pytest: Retry set until failure after ACL update by @abhijat in #5823
• ci: Allow overriding log level for modules when repeating tests by @abhijat in #5824
• feat: SimSIMD update and integration by @vyavdoshenko in #5846
• feat(search): FT.SEARCH FILTER option by @mkaruza in #5830
• refactor: restore stack size for debug builds by @BorysTheDev in #5857
• feat: add FT.CONFIG command by @BorysTheDev in #5855
• chore(deps): bump the actions group with 9 updates by @dependabot[bot] in #5841
• chore(deps): bump gopkg.in/yaml.v3 from 3.0.0 to 3.0.1 in /tools/replay by @dependabot[bot] in #5850
• chore(deps): bump golang.org/x/sys from 0.0.0-20210330210617-4fbd30eecc44 to 0.1.0 in /tools/replay by @dependabot[bot] in #5848
• tests: Add test for prometheus metrics sanity check by @abhijat in #5843
• fix: skip full sync during partial sync by @kostasrim in #5580
• fix: clang build for macos by @vyavdoshenko in #5862
• chore: new ReplicaOf algorithm by @kostasrim in #5774
• chore: update go version by @romange in #5859
• chore(deps): bump github.com/redis/go-redis/v9 from 9.7.0 to 9.7.3 in /tools/replay by @dependabot[bot] in #5847
• fix: crash when WITH_SEARCH=OFF by @BorysTheDev in #5867
• actions: Set node version by @abhijat in #5868
• Clean up blocking by @dranikpg in #5836
• feat: add field prefix/suffix search by @BorysTheDev in #5871
• fix: SimSIMD dynamic dispatch by @vyavdoshenko in #5870
• fix: allow 5% tolerance if snapshot used memory greater than maxmemory limit by @BorysTheDev in #5869
• feat(search): Implement GEO index and support for RADIUS search by @mkaruza in #5854
• fix: Fix Fedora 30 and MacOS build by @mkaruza in #5880
• chore(hset_family): Code cleaning by @dranikpg in #5882
• core: Add data structure for estimate which decays over time by @abhijat in #5879
• fix(search): Ensure RemoveDoc before SORT STORE overwrite to keep indices consi stent by @vyavdoshenko in #5887
• fix: Add type safety to FindMutable calls by @vyavdoshenko in #5886
• refactor: Add DelMutable() helper to unify post_updater.Run() + Del() pattern by @vyavdoshenko in #5890
• Fix: environment variable typo in comment by @maets in #5866
• feat: add symbols search in tag by @BorysTheDev in #5895
• feat(search): Add DD option support for FT.DROPINDEX command by @vyavdoshenko in #5885
• chore(hset_family): Listpack wrapper by @dranikpg in #5889
• fix: SINTERSTORE/SUNIONSTORE/SDIFFSTORE crash when overwriting indexed keys by @vyavdoshenko in #5903
• fix: ZINTERSTORE/ZUNIONSTORE crash when overwriting indexed HASH/JSON keys by @vyavdoshenko in #5902
• server: Remove unused variable and field by @abhijat in #5904
• fix(evicition): Don't accumulate deleted bytes if there is no RSS eviction by @mkaruza in #5894
• chore(server): Small dead code removal by @dranikpg in #5907
• feat(hset_family): create HMapWrap by @dranikpg in #5900
• chore(deps): bump actions/setup-node from 4 to 5 in the actions group by @kostasrim in #5909
• server: Support RESP3 map response for xread by @abhijat in #5898
• fix(docker): prevent zombie processes by @vyavdoshenko in #5878
• chore(deps): bump actions/setup-node from 5 to 6 in the actions group by @dependabot[bot] in #5910
• fix(search): prevent crash on HSET with same key across databases by @vyavdoshenko in #5913
• fix(stream): prevent assert failure in XDEL with non-existent ID by @vyavdoshenko in #5918
• feat: json debug memory command added by @vyavdoshenko in #5884
• feat(tiering): Decoders by @dranikpg in #5883
• fix: Allow JSON.DEBUG HELP to work without key parameter by @vyavdoshenko in #5919
• feat(server): Add shutdown watchdog by @abhijat in #5915
• chore(server): Better hash tests (Part 1) by @dranikpg in #5922
• server: Yield fiber if running for over 1ms by @abhijat in #5924
• feat(tiering): Serialized map by @dranikpg in #5896
• feat(core): External representation enum by @dranikpg in #5912
• test: fix shutdown_test by waiting for server to be ready after restart by @vyavdoshenko in #5930
• chore(hset_family): Add mutable functions to listpack wrapper by @dranikpg in #5901
• chore: code redundancies in replication metrics by @kostasrim in #5926
• fix: Handle non-STRING types in ParseRedis to prevent crash by @vyavdoshenko in #5937
• chore(server): Allow using background fibers for DEBUG OBJHIST by @dranikpg in #5933
• chore: non preemptive ProtocolClient destructor by @kostasrim in #5927
• chore(hset): Mutable HMapWrap functions by @dranikpg in #5928
• feat(tools): Vector search benchmark by @dranikpg in #5938
• search: Return a default_score with ft.info index definition by @abhijat in #5941
• server: Exit consume-redis-stream loop on error by @abhijat in #5948
• chore: reduce frequency of clock invocations by @romange in #5946
• fix(set_family): Don't replicate SPOP if nothing was removed by @mkaruza in #5936
• feat: Add AFL++ fuzzing integration with persistent mode by @vyavdoshenko in #5932
• fix: reject non-STRING arguments in RESP protocol to prevent crash by @vyavdoshenko in #5955
• fix: preserve correct command order in MULTI/EXEC transactions by @vyavdoshenko in #5954
• chore: reduce smoothing interval for local monitoring by @romange in #5965
• fix(tiering): Remove disk store files on exit by @dranikpg in #5966
• fix(tiering): Limit offload budget by time by @dranikpg in #5970
• U...

This is a patch release

It is strongly advised to update to this version if you use 1.34.0 / 1.34.1 due to regression when Dragonfly runs in cache mode

What's Changed

• fix(evicition): Don't accumulate deleted bytes if there is no RSS evi… by @BorysTheDev in #5908

Full Changelog: v1.34.1...v1.34.2

This is a patch release

Known issues

#5899
#5891

What's Changed

• fix: prometheus is unable to scrape the /metrics endpoint #5829

Full Changelog: v1.34.0...v1.34.1

Dragonfly v1.34.0

Known issues

#5899
#5891

Important stability fixes and changes:

• Replication fixes: #5727 #5700
• Geo search fixes: #5714 #5678 #5726 #5734
• SSD tiering improvements: #5802 #5803 #5810

What's Changed

• fix: prevent writeBucket if migration is canceled by @BorysTheDev in #5676
• chore: group memory prometheous metrics by @kostasrim in #5674
• fix: check snapshot_version in correct place by @BorysTheDev in #5687
• fix: stack corruption in MultiCommandSquasher by @romange in #5697
• fix: RdbSaver: destroy SliceSnapshot on its shard thread to fix flaky crash on ARM64 with --force_epoll by @vyavdoshenko in #5685
• server: fix max emission limit in throttle command by @abhijat in #5686
• fix: list ops transformation during replication by @BorysTheDev in #5700
• fix: command parsing in geo family by @kostasrim in #5714
• chore: add queue task index to the log by @romange in #5716
• chore(server): Semi-automatic flag updates by @dranikpg in #5666
• feat(search): Add indexes joining by @BagritsevichStepan in #5604
• feat(search_family): Add LOAD_FROM to the FT.AGGREGATE command by @BagritsevichStepan in #5703
• feat(search_family): Improve sorting and limiting in index joins by @BagritsevichStepan in #5717
• fix: macos and linux build by @vyavdoshenko in #5718
• feat: Implement GEORADIUS_RO command by @EricHayter in #5678
• test: skip failing tests by @vyavdoshenko in #5721
• docs: improve description of --maxmemory parameter (units/tiering/minPerThread) by @frivoire in #5696
• chore: update task queue fiber priority to high by @romange in #5722
• chore: check fail for empty listpacks in streams by @kostasrim in #5704
• feat: improve target node latency during migration by @BorysTheDev in #5715
• chore: Strict bounds when adding uknown command for INFO ALL stats by @mkaruza in #5735
• server: Drop contents of exec_info body on clear by @abhijat in #5740
• feat: Implement GEORADIUSBYMEMBER_RO command by @EricHayter in #5726
• fix: reject geosearch count 0 by @kostasrim in #5734
• fix: jsonPath parser for int values by @BorysTheDev in #5737
• feat(dfly_bench): Generate value with random length by @mkaruza in #5743
• fix: JSON RESP3 compatibility - remove extra nested arrays for most commands by @vyavdoshenko in #5747
• chore: rewrite mimalloc2 external lib dependency by @romange in #5751
• fix: replication for Z...STORE commands by @BorysTheDev in #5728
• server: Enable lua legacy float response for RESP3 by @abhijat in #5754
• fix: bitfield no operations command by @BorysTheDev in #5756
• test: unskip epoll cluster and replication tests by @BorysTheDev in #5757
• Pull helio by @dranikpg in #5760
• fix: search index restore when index names start with ‘:’ (e.g. :Order:index) by @vyavdoshenko in #5755
• Revert "test: unskip epoll cluster and replication tests (#5757)" by @BorysTheDev in #5761
• chore: add more logs for takeover by @BorysTheDev in #5765
• chore: implement the iterative fragmentation check by @romange in #5766
• server: Adds metrics for background defrag task by @abhijat in #5767
• server: metrics to track stored command sizes by @abhijat in #5764
• chore: unlink deletes hashes asynchronously by @romange in #5773
• fix: acl pubsub should only glob match the channel by @kostasrim in #5769
• chore: add self-testing code for heap->full_page_size computation by @romange in #5772
• chore: tune replication/migration settings by @romange in #5779
• facade/server: Use inline instead of extern to reduce loc by @abhijat in #5768
• fix: script error reply by @BorysTheDev in #5776
• chore: Resize vector in StringMap::RandomPairs by @mkaruza in #5781
• dfly_bench: Show error on failing to make address by @abhijat in #5787
• feat: SHUTDOWN: fast path for NOW/FORCE; unify SAVE/SAFE; support NOSAVE by @vyavdoshenko in #5783
• core: Replace sbf with hyperloglog by @abhijat in #5784
• server: Reject caching command if tracking is not enabled by @abhijat in #5786
• fix: data loss in partial sync from same source master by @kostasrim in #5727
• fix: Scan on non-existing key should return zero cursor by @mkaruza in #5780
• fix(snapshot): Replace sleeps with yield by @dranikpg in #5619
• fix(block_list): Speed up Split method by @BagritsevichStepan in #5748
• chore: pass to a string compact_object a key/value flag by @romange in #5795
• feat(range_tree): Add FinalizeInitialization method to the RangeTree by @BagritsevichStepan in #5793
• fix: stale pointer reference during slot migration and flush by @romange in #5798
• chore: Command category cleanup (part 2?) by @dranikpg in #5800
• fix: remove limitation for SSCAN by @BorysTheDev in #5806
• chore: Update helio submodule by @mkaruza in #5807
• chore(tiering): Fix asserts around reads for serialization by @dranikpg in #5802
• feat: prometheus metrics per database by @qlimenoque in #5770
• chore(rdb): Insert big values only after full construction by @dranikpg in #5796
• fix(tiering): Cooldown for DiskStorage::Grow() retries by @dranikpg in #5803
• ci: enable repeat tests to use release and force epoll by @abhijat in #5805
• chore: Clean up string_family by @dranikpg in #5801
• chore(tiering): Make lower bound configurable by @dranikpg in #5810
• fix: increase fiber stack by @BorysTheDev in #5817
• chore: support huffman encoding for string values by @romange in #5815
• chore: increase timeout to 80mins for reg tests by @kostasrim in #5819
• chore(server): Hide internal CO flags from output by @dranikpg in #5814
• feat: Add eviction based on rss memory usage by @BagritsevichStepan in #5791

Huge thanks to all the contributors! ❤️

New Contributors

• @frivoire made their first contribution in #5696
• @qlimenoque made their first contribution in #5770

Full Changelog: v1.33.0...v1.34.0

This is a patch release.

It is strongly advised to update to this version if you use 1.33 due to regression below.

What's Changed

• fix : stack corruption in MultiCommandSquasher #5697
  It also fixed the io_uring compatibility issue with older kernels (#5693)

Dragonfly v1.33.0

Important stability fixes and changes:

• Fix stability bugs in georadius #5629 #5634
• Cluster related improvements: #5614 #5621 and #5618

What's Changed

• server: Log warning if server stops accepting connections by @abhijat in #5596
• feat(server): support command help command by @guozhihao-224 in #5601
• chore: refactor JournalItem by @kostasrim in #5585
• feat(linear_search_map): Add keys method by @BagritsevichStepan in #5603
• core: Minor cleanups in bloom filters by @abhijat in #5595
• fix(tiering): Block on concurrent backing file grow by @dranikpg in #5609
• feat: use SET instead of RESTORE for strings during slot migrations by @BorysTheDev in #5614
• chore: introduce shard_thread_busy_polling_usec to control busy polling by @romange in #5624
• fix: UB in georadius by @BorysTheDev in #5629
• fix: test_throttle_on_commands_squashing_replies_bytes by @kostasrim in #5628
• chore: more pipeline latency coverage by @romange in #5632
• fix: ub in GEORADIUSBYMEMBER by @kostasrim in #5634
• chore: more pipeline latency coverage (#5632) by @romange in #5635
• feat: use PXAT flag instead of separate command for strings during migration by @BorysTheDev in #5631
• refactor: CmdArgParser::Error() by @BorysTheDev in #5636
• test(acl): Fix flaky ACL tests with long-running Lua scripts by @vyavdoshenko in #5627
• chore: string_view overload for BaseFamilyTest::Run by @kostasrim in #5637
• fix: FlushSlots test by @BorysTheDev in #5639
• fix: macos build geo_family.cc by @vyavdoshenko in #5640
• fix: preserve null characters in hash values after HEXPIRE by @vyavdoshenko in #5642
• test(acl): make ACL tests with concurrent scripts deterministic by @vyavdoshenko in #5641
• test: skip test_acl_del_user_while_running_lua_script becuase it constantly fails on CI by @vyavdoshenko in #5646
• chore: do not shutdown after takeover only in full cluster mode by @kostasrim in #5621
• chore: add more I/O run-time settings by @romange in #5644
• fix: Prevent race condition during concurrent SAVE operations by @vyavdoshenko in #5633
• core: Track objects skipped during defragmentation by @abhijat in #5647
• chore(tiered_storage): Rename flags, make them updatable by @dranikpg in #5608
• chore: allow replicaof in cluster mode when state is TAKEN_OVER by @kostasrim in #5618
• fix: timeout overflow during migration finalization by @BorysTheDev in #5652
• fix(json): support json keys containing dots in bracket notation by @vyavdoshenko in #5650
• fix: accounting of metrics under squash_stats_latency_lower_limit filter by @romange in #5659
• chore: sort per shard sorted sets for SINTER by @kostasrim in #5617
• fix: macos build tiered_storage.h by @vyavdoshenko in #5663
• fix: RoundRobinSharder for all proactors by @kostasrim in #5654
• workflows: Add a dispatch action which will run pytests on repeat by @abhijat in #5668
• fix(pytest): Don't delete and modify huge value keys in test_replication_all by @mkaruza in #5657
• chore: move tiering flags by @kostasrim in #5665
• fix(json_family): Remove expiry of existing keys in JSON.SET by @mkaruza in #5670
• fix: test_migration_timeout_on_sync by @BorysTheDev in #5667
• fix: refactor save synchronization to prevent deadlocks and crashes by @vyavdoshenko in #5655
• fix(pytest): Increase timeout in test_seeder_key_target by @mkaruza in #5677
• chore: calibrate regression tests running time by @kostasrim in #5658
• chore(lua): Add lua redis.log function by @mkaruza in #5672
• chore: add more detailed timings for slow squash hops by @romange in #5679

Full Changelog: v1.32.0...v1.33.0

Dragonfly v1.32.0

Important stability fixes and changes:

• Lots of performance improvements around Search #5341#5327 #5350 #5545 #5533
• Add optional latency percentile stats #5372 and #5428
• Update mimalloc version to 2.24
• Added BZMPOP and BLMPOP commands #5436 and #5370
• Added CONFIG REWRITE command #5471

What's Changed

• test(search): add unified benchmarks for prefix/suffix/infix search by @vyavdoshenko in #5319
• chore: get rid of quicklist encoding by @romange in #5282
• feat: add flag pause_wait_timeout by @BorysTheDev in #5328
• test: enable test_multidim_knn python test by @vyavdoshenko in #5329
• test: make random determenistic in test_multidim_knn python test by @vyavdoshenko in #5335
• fix: data race when doing a snapshot in cache mode by @romange in #5336
• chore(search_family): Minor performance improvements to GetAllResults methods. FIRST PR by @BagritsevichStepan in #5341
• fix: JSON.DEL to return 0 for non-existing keys instead of error by @vyavdoshenko in #5343
• chore: improve test_full_sync test by @romange in #5340
• chore: Refactor CompactObject encoding by @dranikpg in #5342
• feat: Added Sort_RO Command. by @H4R5H1T-007 in #5332
• chore: avoid deduplicating docs ids when not needed by @romange in #5344
• chore: add hdr_histogram to the project by @romange in #5330
• chore(tiering_test): tune memory settings to be below 6GB by @romange in #5345
• chore: suffix search by @dranikpg in #5327
• test: Vector operations benchmark was added by @vyavdoshenko in #5353
• feat(replication): Aggregate data until threshold before replication by @mkaruza in #5308
• SimSIMD integration for Vector operations by @vyavdoshenko in #5348
• feat (server) : introduce no point in time replication by @adiholden in #5103
• chore: move 3rd party libraries into a separate cmake file by @romange in #5362
• fix(pytest): Increase allowed diff in test_partial_replication_on_same_source_master by @mkaruza in #5363
• fix(replication): reduce ERROR logging during normal shutdown by @vyavdoshenko in #5366
• chore: remove unneeded mimalloc-specific code by @romange in #5367
• chore: add mimalloc2 third_party library by @romange in #5364
• fix(replication): reduce ERROR to WARNING for connection issues during sync by @vyavdoshenko in #5369
• feat(search): Suffix/Infix search for Tags by @dranikpg in #5350
• fix(zset): expand parsing of rank arguments beyond int32 by @romange in #5377
• test: replication_test.py::test_bug_in_json_memory_tracking enabled by @vyavdoshenko in #5375
• chore: Disable test_replication_all cache mode by @mkaruza in #5385
• chore: Make fakeredis part of the core CI checks by @romange in #5380
• fix(zset_family): Return early if range is expected to be zero by @mkaruza in #5386
• fix: JSON.STRLEN behavior for non-existent keys by @vyavdoshenko in #5389
• chore: add psync metrics for replica by @kostasrim in #5365
• fix(server): fix serialization on move with big values by @adiholden in #5390
• feat(pytest): Set and log random seed used in pytest by @mkaruza in #5391
• chore: remove zset/t_zset files by @romange in #5387
• feat: add support for BLMPOP command by @EricHayter in #5370
• feat(pytest): Initialize random seed for each test by @mkaruza in #5395
• fix(values): fix typos in Helm README and values file by @adimoragbloch in #5400
• fix: no set/hmap expiration during serialization by @BorysTheDev in #5349
• feat(search_family): Add SortedVector<pair<DocId, double>> by @BagritsevichStepan in #5405
• feat: add snapshot-id field into dfs files by @BorysTheDev in #5399
• chore: Enable point_in_time_snapshot by default by @mkaruza in #5403
• chore: Increase BLMPopBlocking sleep duration by @mkaruza in #5408
• chore: skip flaky test_pipeline_cache_size by @kostasrim in #5410
• chore: introduce class scope fixtures by @romange in #5409
• fix(pytest): Increase REPLTAKOVER time in test_take_over_seeder by @mkaruza in #5415
• server_family: Expose command latency stats as part of info by @abhijat in #5372
• fix(search): Cut off results before serializing by @dranikpg in #5412
• test: Add Dragonfly search benchmark tests by @vyavdoshenko in #5406
• feat(search_family): Introduce RangeTree by @BagritsevichStepan in #5414
• fix: search_benchmark_utils on python 3.12 by @romange in #5422
• feat(search_family): Add RangeTree to NumericIndex by @BagritsevichStepan in #5416
• chore: switch to mimalloc2 by @romange in #5368
• feat(search_family): Add optimized merging logic for one and two blocks in RangeResult by @BagritsevichStepan in #5417
• chore(range_tree): Simplify RangeTree code by @BagritsevichStepan in #5418
• Feat config management by @boomballa in #5388
• Lua library Integer overflow can cause the DragonFly crash. (CVE-2020-14147) by @KIMDONGYEON00 in #5421
• chore(search): LIMIT=0 Fast path and remove LOAD parameter from FT.SEARCH by @dranikpg in #5419
• server_family: Support resetting latency histogram by @abhijat in #5428
• test: search benchmark updated to use class fixture for multiple tests by @vyavdoshenko in #5425
• chore: harden checks in _mi_page_malloc_zero by @romange in #5427
• fix(pytest): Wait replication to be in sync in test_stream_approximate_trimming by @mkaruza in #5430
• chore: avoid squashing when squashing_current_reply_size crosses limit by @kostasrim in #4924
• fix: acl log missing return after reply by @kostasrim in #5454
• chore: add acl help by @kostasrim in #5456
• chore: allow initializing a separate huffman table for string values by @romange in #5470
• feat(search_family): Introduce SingleBlockRangeResult and TwoBlocksRangeResult by @BagritsevichStepan in #5424
• chore: add tls_caching flags by @kostasrim in #5485
• chore: add more logs during replication by @BorysTheDev in #5489
• chore(set_family): Fast path SPOP with small count by @dranikpg in #5491
• fix: add a sleep to relax slot migration process by @romange in #5478
• chore: add tls_socket accept error metric by @romange in #5492
• chore: add io_read_cnt metric by @romange in #5494
• chore: add tls handshake statistics by @kostasrim in #5493
• feat: add more auxillary strings to the snapshot by @romange in #5411
• chore: log frequency for IsReplySizeOverLimit by @kostasrim in #5501
• fix: adjust BLMPOP command options and implementation for proper journaling by @EricHayter in #5503
• chore: add more metrics by @romange in https://github.com/dragonflydb/dragonfly/p...

This is a patch release.

What's Changed

• fix: prevent idle disconnects for replica flow connections (#5506)
• feat: add debug segments command (#5505)
• server: Make memory defrag blocking, accept optional threshold (#5498)
• fix: allow eviction of zero-allocation entries in cache mode (#5500)
• chore: add more metrics (#5504)

Full Changelog: v1.31.1...v1.31.2

This is a patch release.

What's Changed

• fix: two tls issues
  1. Excessive logging during tls handshake in some cases.
  2. TLS Short read that hinders read performance.
• chore: add tls handshake statistics (#5493)
• chore: add io_read_cnt metric (#5494)
• chore: add tls_socket accept error metric (#5492)
• fix: add sleep to relax slot migration process (#5478)
• fix(set_family): Fast path SPOP with small count (#5491)
• chore: add more logs during replication (#5489)
• chore: add tls_caching flags (#5485)

Full Changelog: v1.31.0...v1.31.1