mirrors/dragonfly
1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2025-05-11 18:35:46 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

feat: allow throttling tiered writes (#2414)

Browse source 
* feat: allow throttling tiered writes

The throttling is controlled by tiered_storage_throttle_us flag
and can be disabled by passing `--tiered_storage_throttle_us=0`.
This introduces a soft back-pressure during writes.

On my machine `debug POPULATE 10000000 key 1000 RAND` with tiered_storage_throttle_us=0
offloads 12% of all the entries, but with tiered_storage_throttle_us=1 it offloads
almost 100% by prolonging the operation from 0.96s to 1.72s.
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
This commit is contained in:
Roman Gershman 2024-01-15 13:21:23 +02:00 • committed by GitHub
parent f4c1e33d48
commit b6f4370ae7
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
9 changed files with 50 additions and 24 deletions
Download patch file Download diff file Expand all files Collapse all files

9
src/server/common.h
View file

@ -123,15 +123,16 @@ void RecordExpiry(DbIndex dbid, std::string_view key);
void TriggerJournalWriteToSink();
struct TieredStats {
size_t tiered_reads = 0;
size_t tiered_writes = 0;
uint64_t tiered_reads = 0;
uint64_t tiered_writes = 0;
size_t storage_capacity = 0;
// how much was reserved by actively stored items.
size_t storage_reserved = 0;
size_t aborted_write_cnt = 0;
size_t flush_skip_cnt = 0;
uint64_t aborted_write_cnt = 0;
uint64_t flush_skip_cnt = 0;
uint64_t throttled_write_cnt = 0;
TieredStats& operator+=(const TieredStats&);
};