Before - we used a global atomic var current_reply_size that was constantly updated by all threads
when reply buffers grew for squashing.
Now, we use per thread atomic variables that track reply buffer size for the I/O thread that issues squashing.
The shard threads contend less because they update multiple atomic variables. Moreover,
now we can adjust IsPipelineBufferOverLimit to take into account squashing_current_reply_size as well.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Before - we used a global atomic var current_reply_size that was constantly updated by all threads
when reply buffers grew for squashing.
Now, we use per thread variables. The disadvantage - the metric is less precise because we first allocate buffers
in a shard thread but update the thread local metrics in the coordinator threads.
I think the tradeoff is fair and the fact that the metric is updated with delay is not crucial.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
1. Remove one vector (affects allocation and data locality) in squashing.
2. stop deduplicating MGET keys by default, but keep it as a run-time flag.
Also, finally fix TieredStorageTest.FlushAll test.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Before: StoredCmd always copied the backing buffer of the commands.
this of course sub-optimal if the bucking buffer exists during the life-time
of StoredCmd. This is exactly the case in `Service::DispatchManyCommands`.
This PR:
1. Adds support for both owned and non-owned arguments.
2. Improves the interfaces around StoredCmd and removes some code duplication.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
1. Eliminate replies reverse call - will allow to unite replies and cmd vectors into one.
2. Introduce SwitchTxCmd function to avoid code duplication.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* rename StaticSeeder to DebugPopulateSeeder to show intent
* add a DCHECK in debug populate that triggers if there is a registered replica that hasn't reached stable sync
* small cleanup
Signed-off-by: kostas <kostas@dragonflydb.io>
* feat(cluster): Add `RestoreStreamer`.
`RestoreStreamer`, like `JournalStreamer`, streams journal changes to a
sink. However, in addition, it traverses the DB like `RdbSerializer` and
sends existing entries as `RESTORE` commands.
Adding it required a bit of plumbing to get all journal changes to be
slot-aware.
In a follow-up PR I will remove the now unneeded `SerializerBase`.
* Fix build
* Fix bug
* Remove unimplemented function
* Iterate DB, drop support for db1+
* Send FULL-SYNC-CUT
