The bug was caused by incorrect handling of corner cases,
when a path that should lead to an item was wrongfully cleared, which lead
to empty results for SortedMap::GetRange query.
This PR:
1. fixes the wrong code in bptree_set.h.
2. Adds unit tests for both bptree_set_test and sorted_map_test.
Related to https://github.com/mastodon/mastodon/issues/33805
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
1. Add testdata, move the existing test data to zstd-compressed format.
2. Add benchmarks in qlist_test that add lots of records, forcing compression of inner noded,
or read compressed list checking the decompression.
Currently qlist uses valkey compression algorithm.
On my machine:
1. (valkey) Compression is 5 times slower than no compression.
2. Decompression is 37 times slower.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Also fix sentinel test by using a precise redis-server version.
Finally, add pytest warnings filter to reduce noise
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
1. Make GlobMatcher non-copyable. It's something I will need in the next PR.
2. Move low-level benchmarking testing code into dfly_core_test.
No functional changes.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: Add Lua force atomicity flag
We accidentally instructed Sidekiq to add `disable-atomicity` to their
script, despite them needing to run atomically.
This hack-ish PR adds a `--lua_force_atomicity_shas` flag to allow
specifying which SHAs are forced to run in an atomic fashion, even if
they are marked as non-atomic.
Fixes#4522
* fix build on clang
fix: Do not bump elements during RDB load #4507
The Issue
Before this PR, when loading an RDB, we modified fetched_items_ as part of the loading process. This has little effect, unless the next issued command calls FLUSHALL / FLUSHDB (could happen in DFLY LOAD, REPLICAOF or just calling FLUSHALL directly). In such a case, a CHECK() fails.
The Fix
While load is not run as a command (in a transaction), it still uses APIs that assume that they are called in the context of a command. As such, it indirectly used DbSlice::FindInternal(), which bumps elements when called.
This PR adds another sub-mode to DbSlice, named load_in_progress_. When true, we treat DbSlice as if it is not in cache mode, ignoring cache_mode_.
BTW this PR also renames caching_mode_ to cache_mode_ as we generally use the term cache mode and not caching mode, including in the --cache_mode flag.
Fixes#4497
Our previous weekly pipeline used qemu, was very slow and over-complicated.
This one uses matrix with proper parallelization and the latest arm64 github runners.
now it takes less than 30 minutes to build everything.
lets make it daily.
1. Fixes#4466 (we did not check validity of the scan cursor)
We fix it by quietly returning "end of scan" result, i.e. no error are returned.
2. Add 34b1048274c8e50a0cc587a3ed9c383a82bb78c5 sha to the lua_undeclared_keys_shas list.
3. Pull helio that fixes bugs with WaitQueue and IdleTasks: 2ab4412b78
* feat: add 'testing_time' limit option to dfly_bench
---------
Signed-off-by: Roman Gershman <romange@gmail.com>
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
feat: add support for noreply and greeat options
noreply option is applicable for memcached.
great option ensure the tcp connection is indeed established before proceeding with loadtest.
Before this PR, QueueBackpressure objects are in fact referenced from other threads in an awkward way via
Connection::WeakRef::EnsureMemoryBudget().
This PR removes the complexities of accessing these objects from foreigh threads.
* bump up redis version in info command
* add compatibility test
* bump up py dependencies
* fix warnings and deprecated functions
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
* chore: dfly_bench now accepts __data__ and __score__ template names
---------
Signed-off-by: Roman Gershman <romange@gmail.com>
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
