mirrors/dragonfly
1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2025-05-11 10:25:47 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

chore: introduce a cool queue that gradually retires cool items (#3377)

Browse source 
* chore: introduce a cool queue that gradually retires cool items

This PR introduces a new state in which the offloaded value is not freed from memory but instead stays
in the cool queue.

Upon Read we convert the cool value back to hot table and delete it from storage.
When we low on memory we retire oldest cool values until we are above the threshold.

The PR does not fully finish the feature but it is workable enough to start (load)testing.
Missing:
a) Handle Modify operations
b) Retire cool items in more cases where we are low on memory. Specifically, refrain from evictions as long as cool items exist.

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
This commit is contained in:
Roman Gershman 2024-07-25 09:09:40 +03:00 • committed by GitHub
parent 02b72c9042
commit 8a9c9adbc5
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 220 additions and 43 deletions
Download patch file Download diff file Expand all files Collapse all files

16
src/server/engine_shard_set.cc
View file

@ -387,12 +387,9 @@ EngineShard::~EngineShard() {
}
void EngineShard::Shutdown() {
queue_.Shutdown();
DVLOG(1) << "EngineShard::Shutdown";
if (tiered_storage_) {
tiered_storage_->Close();
tiered_storage_.reset();
}
queue_.Shutdown();
DCHECK(!fiber_periodic_.IsJoinable());
@ -894,7 +891,14 @@ void EngineShardSet::Init(uint32_t sz, bool update_db_time) {
}
void EngineShardSet::PreShutdown() {
RunBlockingInParallel([](EngineShard* shard) { shard->StopPeriodicFiber(); });
RunBlockingInParallel([](EngineShard* shard) {
shard->StopPeriodicFiber();
// We must close tiered_storage before we destroy namespaces that own db slices.
if (shard->tiered_storage()) {
shard->tiered_storage()->Close();
}
});
}
void EngineShardSet::Shutdown() {