When expiry is set on an hset its members are migrated to a string map.
If a scan is performed on the hset, the code branch for string map
accessed the value pointer directly by loading the 64 bit value from the
object pointer.
When creating a new entry in string map with TTL, we also set a bit on
the stored pointer as metadata, this is set as 1 << 63.
As a result when loading back the pointer we need to unset the same bit
to get the correct address, this is done already in string map internals
but the scan code did not do this, resulting in segfault.
This change adds an AND operation to unset the TTL bit before
dereferencing the pointer.
Signed-off-by: Abhijat Malviya <abhijat@dragonflydb.io>
thread_queue_backpressure is a global array of per thread QueueBackpressure
objects. We referenced these objects incorrectly in 1.27.0-2.
Fixes#4770
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
The bug is that expiring keys during heartbeat should not preempt while writing to the journal and we assert this with a FiberAtomicGuard. However, this atomicity guarantee is violated because the journal callback acquires a lock on a mutex that is already locked by on OnJournalEntry(). The fix is to release the lock when OnJournalEntry() preempts.
Signed-off-by: kostas <kostas@dragonflydb.io>
Before this PR, EVAL transactions would always use db_index=0
leading to inconsistent interactions with other commands on non-zero
databases. Specifically the test in the PR would deadlock.
This PR fixes the issue. Also fixes#4583
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
1. Rename FinalizeWatched to RemovedWatched.
2. Simplify passing of WatchedKeys - instead of passing a callback
we pass either a single value or signal that all the shard
keys should be used. Less generic but more explicit.
3. Reverse the result from RunInShard() function - to improve readability.
All in all it should not have any functional changes besides logging.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
The bug requires lots of conditions in order to reproduce:
1. blocking operations on multiple databases
2. use of lua scripts that wake blocking transactions
The bug was discovered due to a deadlock in BLMOVE but could also manifest with other commands that would
"disappear" causing local starvation effects on the connections sending them.
With BLMOVE it causes a global deadlock in the transaction queue in dragonfly.
The fix is actually deleting a few lines of code introduced by #3260 from 6 months ago,
so it is actually a long lived regression.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
1. Provide clear usage instructions
2. Add "pace" option, which when false, sends traffic as quickly as possible (default true).
3. Add skip option that sometimes can be useful to remove unneeded noise
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: Add FakeRedis capture/compare to more cluster tests
Note that we can't add such a test to `test_cluster_native_client`
because we don't do a migration there, just re-assign slots, meaning
that data is lost (by desgin?)
Fixes#4429
* Seed during migrations
* Add TODO
The bug was caused by incorrect handling of corner cases,
when a path that should lead to an item was wrongfully cleared, which lead
to empty results for SortedMap::GetRange query.
This PR:
1. fixes the wrong code in bptree_set.h.
2. Adds unit tests for both bptree_set_test and sorted_map_test.
Related to https://github.com/mastodon/mastodon/issues/33805
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
1. Add testdata, move the existing test data to zstd-compressed format.
2. Add benchmarks in qlist_test that add lots of records, forcing compression of inner noded,
or read compressed list checking the decompression.
Currently qlist uses valkey compression algorithm.
On my machine:
1. (valkey) Compression is 5 times slower than no compression.
2. Decompression is 37 times slower.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Also fix sentinel test by using a precise redis-server version.
Finally, add pytest warnings filter to reduce noise
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
1. Make GlobMatcher non-copyable. It's something I will need in the next PR.
2. Move low-level benchmarking testing code into dfly_core_test.
No functional changes.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: Add Lua force atomicity flag
We accidentally instructed Sidekiq to add `disable-atomicity` to their
script, despite them needing to run atomically.
This hack-ish PR adds a `--lua_force_atomicity_shas` flag to allow
specifying which SHAs are forced to run in an atomic fashion, even if
they are marked as non-atomic.
Fixes#4522
* fix build on clang
