mirrors/dragonfly
1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2025-05-11 10:25:47 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

chore: reenable evictions upon insertion to avoid OOM rejections (#3387)

Browse source 
* chore: reenable evictions upon insertion to avoid OOM rejections

Before: when running dragonfly with --cache_mode we could get OOM rejections
even though the eviction policy allowed to evict items to free memory.
Ideally, dragonfly in cache mode should not respond with the OOM error.

This PR reuses the same Eviction step we have in the Heartbeat and conditionally applies it
during the insertion. In my test the OOM errors went from 500K to 0 and the server
still respected memory limit.

Also, remove the old heuristics that has never been used.

Test:

./dfly_bench --key_prefix=bar: -d 1024 --ratio=1:0 --qps=200 -n 3000
./dragonfly --dbfilename=  --proactor_threads=2 --maxmemory=600M --cache_mode

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
This commit is contained in:
Roman Gershman 2024-07-25 15:28:57 +03:00 • committed by GitHub
parent fb4222d01e
commit e2d65a0900
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 67 additions and 141 deletions
Download patch file Download diff file Expand all files Collapse all files

12
src/server/db_slice.h
View file

@ -437,7 +437,12 @@ class DbSlice {
// Deletes some amount of possible expired items.
DeleteExpiredStats DeleteExpiredStep(const Context& cntx, unsigned count);
void FreeMemWithEvictionStep(DbIndex db_indx, size_t increase_goal_bytes);
// Evicts items with dynamically allocated data from the primary table.
// Does not shrink tables.
// Returnes number of bytes freed due to evictions.
size_t FreeMemWithEvictionStep(DbIndex db_indx, size_t starting_segment_id,
size_t increase_goal_bytes);
void ScheduleForOffloadStep(DbIndex db_indx, size_t increase_goal_bytes);
int32_t GetNextSegmentForEviction(int32_t segment_id, DbIndex db_ind) const;
@ -469,6 +474,8 @@ class DbSlice {
// Resets events_ member. Used by CONFIG RESETSTAT
void ResetEvents();
// Controls the expiry/eviction state. The server may enter states where
// Both evictions and expiries will be stopped for a short period of time.
void SetExpireAllowed(bool is_allowed) {
expire_allowed_ = is_allowed;
}
@ -508,7 +515,6 @@ class DbSlice {
void SendInvalidationTrackingMessage(std::string_view key);
void CreateDb(DbIndex index);
size_t EvictObjects(size_t memory_to_free, Iterator it, DbTable* table);
enum class UpdateStatsMode {
kReadStats,
@ -534,7 +540,7 @@ class DbSlice {
return version_++;
}
void CallChangeCallbacks(DbIndex id, const ChangeReq& cr) const;
void CallChangeCallbacks(DbIndex id, std::string_view key, const ChangeReq& cr) const;
class LocalBlockingCounter {
public: