Commit graph

2248 commits

Author SHA1 Message Date
Roman Gershman
99c4ab8abe
feat: add support for noreply option (#4485)
feat: add support for noreply and greeat options

noreply option is applicable for memcached.
great option ensure the tcp connection is indeed established before proceeding with loadtest.
2025-01-20 13:59:47 +00:00
Borys
010166525e
feat: allow finish and start migration in the same config (#4486) 2025-01-20 13:52:38 +00:00
Mykhailo Faraponov
91435bc6af
Fix typo in outgoing_slot_migration.cc (#4484)
Signed-off-by: Mykhailo Faraponov <11322032+moredure@users.noreply.github.com>
2025-01-20 12:38:02 +00:00
Roman Gershman
7d0530547b
chore: remove pubsub semantics from Connection::WeakRef (#4483)
Make it generic and move pubsub related logic into channel_store.
2025-01-20 12:32:17 +00:00
Roman Gershman
4acafa3d47
chore: OpManagerTest fixes (#4480) 2025-01-20 11:38:26 +00:00
Roman Gershman
d2209d9eea
chore: make per-thread QueueBackpressure objects global (#4482)
Before this PR, QueueBackpressure objects are in fact referenced from other threads in an awkward way via
Connection::WeakRef::EnsureMemoryBudget().

This PR removes the complexities of accessing these objects from foreigh threads.
2025-01-20 12:53:36 +02:00
Kostas Kyrimis
b017cdd1a0
fix: hiredis requires df to report version >7.4 (#4474)
* bump up redis version in info command
* add compatibility test
* bump up py dependencies
* fix warnings and deprecated functions

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-20 10:38:37 +02:00
Shahar Mike
6f3c6e3d57
chore: Fix all clang build warnings (#4475)
* chore: Fix all clang build warnings

Also add `-Werror` to clang build in CI.

Fixes #4449

* all build targets

* fix search test
2025-01-20 10:24:07 +02:00
Kostas Kyrimis
c759eb8ce6
chore: replace deprecated HELIO_STACK_CHECK (#4465)
* remove HELIO_STACK_CHECK
* use FLAGS_fiber_safety_margin

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-20 10:17:24 +02:00
Roman Gershman
ac8e1c80f0
chore: dfly_bench now accepts __data__ and __score__ template names (#4477)
* chore: dfly_bench now accepts __data__ and __score__ template names

---------

Signed-off-by: Roman Gershman <romange@gmail.com>
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
2025-01-20 07:59:03 +00:00
Roman Gershman
a936dfe8a5
chore: add Dash::Prefetch function (#4476)
* chore: add Dash::Prefetch function

It's not being used at this time.

* chore: fixes
2025-01-19 20:07:22 +02:00
Stepan Bagritsevich
99f52642c9
fix(sanitizers): Set stack size to 64 KB during tests for Sanitizers (#4454)
* fix(rdb_test): Fix sanitizers for RdbTest

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* Revert "fix(rdb_test): Fix sanitizers for RdbTest"

This reverts commit 39ae4cf9958c517e70c666caee5a7fdd6beeba0d.

* feat: Add default fiber stack size

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* fix: Increase default stack size

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* fix: Specify default stack size for sanitizers

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* fix: Add SANITIZERS to dfly_test_lib

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

---------

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2025-01-16 10:00:42 +01:00
Borys
0e116b1535
fix: slot calculation during transaction squashing (#4460) 2025-01-15 11:43:52 +00:00
Shahar Mike
5ba608b58d
chore: Add stats print for slot migrations (#4456)
* chore: Add stats print for slot migrations

Fixes #4415
2025-01-15 11:06:09 +00:00
Roman Gershman
e89c15bc6a
fix: resp inline parsing correctly resets itself (#4458)
With empty inline string, the parser failed to continue parsing correctly the non-inline input.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-15 08:45:22 +02:00
Stepan Bagritsevich
f6441df57a
chore: Remove template Callbacks (#4431)
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2025-01-13 15:33:17 +02:00
Roman Gershman
29cde99ca5
chore: pull latest helio (#4446)
This also pulls the latest abseil library 20240722.0
2025-01-13 15:23:08 +02:00
Roman Gershman
7a7b671095
fix: reply_builder should properly serialize bulk strings endings. (#4441)
Due to a corner-case bug, reply builder could add \0\0 to the end of bulk strings, instead of
\r\n. The bug slipped our tests because redis-py parser most probably does not validate the ending
as long as everything else is consistent.

This PR:
1. Adds a test that catches the bug
2. Adds a debug check that verifies the destination pointer is consistent with the iovec being used.
3. Fixes the bug.

Fixes #4424
2025-01-12 12:32:27 +02:00
Roman Gershman
e39e68276e
chore: dragonfly connection refactorings (#4434)
1. Move socket read code into a dedicated function.
   Remove std:: prefix in the code.
2. Add an optional iouring bufring registration. Currently not being used and is disabled by default.
2025-01-09 13:55:00 +00:00
Roman Gershman
f291ae27cb
fix: show managed stuff on privileged port (#4433)
* fix: show managed stuff on privileged port

* chore: comments
2025-01-09 13:56:14 +02:00
Roman Gershman
c77e7cc09f
fix: improve error propagation with RESTORE commands (#4428)
* fix: improve error propagation with RESTORE commands

Also, provide better logs if AddOrNew function fails adding a new entry
2025-01-09 10:22:47 +00:00
Roman Gershman
f3426bd559
chore: update jsoncons version to 0.178 (#4368)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-08 20:05:50 +02:00
Borys
933c9f0b1c
refactor: remove transaction lib on cluster code dependency (#4417) 2025-01-08 09:38:13 +00:00
Kostas Kyrimis
cb752d90cd
chore: split zset and geo family tests (#4425)
* split zset and geo family tests

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-08 09:34:31 +00:00
Roman Gershman
0a4008385d
fix: switch to SHUTTING_DOWN state unconditionally (#4408)
* fix: switch to SHUTTING_DOWN state unconditionally

During the shutdown sequence always switch to SHUTTING_DOWN.
Make sure that the rest of the code does not break if it can not switch to the desired
global state + some clean ups around state transitions.

Finally, reduce the amount of data in test_replicaof_reject_on_load
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-08 11:28:36 +02:00
Kostas Kyrimis
739bee5c83
chore: split geo and zset families (#4421)
* split geo family from zset family

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-08 10:46:25 +02:00
Kostas Kyrimis
27dc1a44f6
fix: geosearch and georadius response format (#4420)
* return array of arrays only when WITHCOORD, WITHDIST, WITHHASH statements are used

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-07 18:15:06 +02:00
Borys
2fe29c39fd
chore: reduce number of logs in cluster tests (#4419) 2025-01-07 16:04:49 +00:00
Shahar Mike
ec6ac23ebc
chore: fix build on FreeBSD (#4412)
* chore: build on FreeBSD

* docs

* older helio
2025-01-07 15:05:50 +02:00
Kostas Kyrimis
e8f43d6858
fix: ExtentTree use of invalidated iterator (#4416)
Fixes accessing and invalidated iterator within ExtendTree
2025-01-07 14:38:32 +02:00
Roman Gershman
a520ce0195
fix: invalid iterator case in Qlist::Erase (#4414)
The problem was with reverse iterator that was not set properly
when the last node is deleted.

Also, move PushSentinel code into Push.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-07 12:56:23 +02:00
Roman Gershman
a4d243b96f
chore: Introduce small buffer in redis parser (#4076)
* chore: Introduce small buffer in redis parser

This is needed in order to eliminate cases where we return INPUT_PENDING but do not consume the whole string by rejecting just several bytes.
This should simplify buffer management for the caller, so that if they pass a string that
did not result in complete parsed request, at least the whole string is consumed and can be discarded.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

* chore: comments fixes

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-07 11:46:14 +02:00
Borys
ed1436bb29
test: move ReplicaofRejectOnLoad test from pytest into unit tests (#4410) 2025-01-07 10:20:36 +02:00
Roman Gershman
21fcf5808e
feat: allow querying of json objects stored as strings (#4399)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-06 21:20:52 +02:00
Shahar Mike
7860a169d9
feat: Yield inside huge values migration serialization (#4197)
* feat: Yield inside huge values migration serialization

With #4144 we break huge values slot migration into multiple commands.
This PR now adds yield between those commands.
It also adds a test that checks that modifying huge values while doing a
migration works well, and that RSS doesn't grow too much.

Fixes #4100
2025-01-05 14:28:45 +00:00
Roman Gershman
ff4add0c9e
chore: deprecate unneeded runtime flags (#4405)
Deprecate multi_exec_mode and track_exec_frequencies.
Remove duplicated call to EnterLameDuck (done also in Service::Shutdown()).

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-05 13:31:59 +00:00
Borys
f663f8e841
fix: provide resp3 option to CapturingReplyBuilder (#4400) 2025-01-05 14:16:02 +02:00
adiholden
92c3749c8c
fix(tests): check cluster big snapshot in unit test (#4403)
fix tests: check cluster big snapshot in unit test

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-05 13:18:13 +02:00
Roman Gershman
4f09fe036c
feat: support deletions with meta protocol (#4398)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-03 13:06:24 +00:00
Roman Gershman
6e9409c65c
fix: properly clear tiered stashes upon expirations (#4395)
Fixes #4387

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-02 13:49:13 +00:00
Borys
0832d23f13
fix: allow cluster node load snapshot bigger than maxmemory (#4394) 2025-01-02 13:50:21 +02:00
adiholden
a3ef239ac7
feat(server): refactor allow preempt on journal record (#4393)
* feat server: refactor allow preempt on journal record

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-02 12:16:21 +02:00
Roman Gershman
7a68528022
chore: minor refactorings around dense_set deletions (#4390)
chore: refactorings around deletions

Done as a preparation to introduce asynchronous deletions for sets/zsets/hmaps.
1. Restrict the interface around DbSlice::Del. Now it requires for the iterator to be valid and the checks should
be explicit before the call. Most callers already provides a valid iterator.

2. Some minor refactoring in compact_object_test.
3. Expose DenseSet::ClearStep to allow iterative deletions of elements.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-02 11:35:55 +02:00
adiholden
3b082e42b8
fix(replication): do not log to journal on callback fail (#4392)
fix replication: do not log to journal on callback fail

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-02 09:39:31 +02:00
guyzilla
413ec0a1cf
feat:Adding support for ZMPOP command (#4385)
Signed-off-by: Guy Flysher <guyzila@gmail.com>
2025-01-01 11:34:53 +02:00
Borys
c5b3584dfd
refactor: slot_set don't use stack memory anymore (#4386) 2025-01-01 10:51:38 +02:00
adiholden
810af83074
fix(server): debug populate consume less memory (#4384)
* fix server: debug populate consume less memory

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-01 10:49:17 +02:00
Roman Gershman
2abe2c0ac2
fix: ExternalAllocator::Free with large sizes (#4388)
ExternalAllocator allocates large sizes directly from the extent tree bypassing segment data structure.
Unfortunately, we forgot to align Free() the same way. This PR:

1. Make sure that we add back the allocated range to the extent tree in Free.
2. Rewrite and simplify ExtentTree::Add implementation.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-12-31 18:41:14 +02:00
adiholden
22994cf3b7
chore(server): cleanup replication shard sync execution (#4374)
chore server: cleanup replication shard sync execution

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-30 11:52:58 +02:00
Borys
c4be62edc8
fix: return value for DflyMigrateAck (#4379) 2024-12-29 09:37:08 +02:00