There are actually a few failures fixed in this PR, only one of which is a test bug:
* `db_slice_->Traverse()` can yield, causing `fiber_cancelled_`'s value to change
* When a migration is cancelled, it may never finish `WaitForInflightToComplete()` because it has `in_flight_bytes_` that will never reach destination due to the cancellation
* `IterateMap()` with numeric key/values overrode the key's buffer with the value's buffer
Fixes#4207
fix 1: in multi command squasher error message was not set therefore it was not printed to log on the relevant command only on exec, fixed by setting the last error in CapturingReplyBuilder::SendError
fix 2: non clearing cached error replies before the command is Invoked
---------
Signed-off-by: adi_holden <adi@dragonflydb.io>
Co-authored-by: kostas <kostas@dragonflydb.io>
* chore: reduce pipelining latency by reusing existing shard fibers
To prove the benefits, run `./dfly_bench --pipeline=50 -n 20000 --ratio 0:1 --qps=0 --key_maximum=1`
Before: the average pipelining latency was 10ms
After: the average pipelining latency is 5ms.
Avg latency: pipelined_latency_usec / total_pipelined_squashed_commands
Also, improved counting of squashed commands - to count actual squashed ones.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* feat(namespaces): Initial support for multi-tenant #3050
This PR introduces a way to create multiple, separate and isolated
namespaces in Dragonfly. Each user can be associated with a single
namespace, and will not be able to interact with other namespaces.
This is still experimental, and lacks some important features, such as:
* Replication and RDB saving completely ignores non-default namespaces
* Defrag and statistics either use the default namespace or all
namespaces without separation
To associate a user with a namespace, use the `ACL` command with the
`TENANT:<namespace>` flag:
```
ACL SETUSER user TENANT:namespace1 ON >user_pass +@all ~*
```
For more examples and up to date info check
`tests/dragonfly/acl_family_test.py` - specifically the
`test_namespaces` function.
* feat(cluster): Add `RestoreStreamer`.
`RestoreStreamer`, like `JournalStreamer`, streams journal changes to a
sink. However, in addition, it traverses the DB like `RdbSerializer` and
sends existing entries as `RESTORE` commands.
Adding it required a bit of plumbing to get all journal changes to be
slot-aware.
In a follow-up PR I will remove the now unneeded `SerializerBase`.
* Fix build
* Fix bug
* Remove unimplemented function
* Iterate DB, drop support for db1+
* Send FULL-SYNC-CUT
fix: fix "debug exec" command
It used mutex lock inside Await callback which is prohibited.
In addition, we improved loggings across the transaction code.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* feat: track differrent patterns of multi/exec transactions
This information is exposed via "DEBUG EXEC" command.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: address comments + add basic squasher stats
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
feat: ignore MULTI/EXEC if the transaction consists of EVAL commands.
Together with `default_lua_config` solves #781.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Make blpop test pass with squashing. The problem was that the txid was not properly reported
back to the testing code.
Also, fixed redundant codepath for defragmentation.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
chore: remove cmd name from the list of arguments
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Roman Gershman <roman@dragonflydb.io>
