Commit graph

3035 commits

Author SHA1 Message Date
Roman Gershman
e89c15bc6a
fix: resp inline parsing correctly resets itself (#4458)
With empty inline string, the parser failed to continue parsing correctly the non-inline input.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-15 08:45:22 +02:00
Stepan Bagritsevich
f6441df57a
chore: Remove template Callbacks (#4431)
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2025-01-13 15:33:17 +02:00
Roman Gershman
29cde99ca5
chore: pull latest helio (#4446)
This also pulls the latest abseil library 20240722.0
2025-01-13 15:23:08 +02:00
romange
679df5cd81 chore(helm-chart): update to v1.26.1 2025-01-13 07:18:17 +00:00
Roman Gershman
7a7b671095
fix: reply_builder should properly serialize bulk strings endings. (#4441)
Due to a corner-case bug, reply builder could add \0\0 to the end of bulk strings, instead of
\r\n. The bug slipped our tests because redis-py parser most probably does not validate the ending
as long as everything else is consistent.

This PR:
1. Adds a test that catches the bug
2. Adds a debug check that verifies the destination pointer is consistent with the iovec being used.
3. Fixes the bug.

Fixes #4424
2025-01-12 12:32:27 +02:00
Roman Gershman
e39e68276e
chore: dragonfly connection refactorings (#4434)
1. Move socket read code into a dedicated function.
   Remove std:: prefix in the code.
2. Add an optional iouring bufring registration. Currently not being used and is disabled by default.
2025-01-09 13:55:00 +00:00
Roman Gershman
f291ae27cb
fix: show managed stuff on privileged port (#4433)
* fix: show managed stuff on privileged port

* chore: comments
2025-01-09 13:56:14 +02:00
Roman Gershman
c77e7cc09f
fix: improve error propagation with RESTORE commands (#4428)
* fix: improve error propagation with RESTORE commands

Also, provide better logs if AddOrNew function fails adding a new entry
2025-01-09 10:22:47 +00:00
Roman Gershman
f3426bd559
chore: update jsoncons version to 0.178 (#4368)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-08 20:05:50 +02:00
Shahar Mike
5adb976207
chore: Increase migration wait time in test_cluster_migration_huge_container (#4427) 2025-01-08 11:40:40 +00:00
Borys
933c9f0b1c
refactor: remove transaction lib on cluster code dependency (#4417) 2025-01-08 09:38:13 +00:00
Kostas Kyrimis
cb752d90cd
chore: split zset and geo family tests (#4425)
* split zset and geo family tests

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-08 09:34:31 +00:00
Roman Gershman
0a4008385d
fix: switch to SHUTTING_DOWN state unconditionally (#4408)
* fix: switch to SHUTTING_DOWN state unconditionally

During the shutdown sequence always switch to SHUTTING_DOWN.
Make sure that the rest of the code does not break if it can not switch to the desired
global state + some clean ups around state transitions.

Finally, reduce the amount of data in test_replicaof_reject_on_load
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-08 11:28:36 +02:00
Kostas Kyrimis
739bee5c83
chore: split geo and zset families (#4421)
* split geo family from zset family

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-08 10:46:25 +02:00
Kostas Kyrimis
27dc1a44f6
fix: geosearch and georadius response format (#4420)
* return array of arrays only when WITHCOORD, WITHDIST, WITHHASH statements are used

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-07 18:15:06 +02:00
Borys
2fe29c39fd
chore: reduce number of logs in cluster tests (#4419) 2025-01-07 16:04:49 +00:00
Shahar Mike
ec6ac23ebc
chore: fix build on FreeBSD (#4412)
* chore: build on FreeBSD

* docs

* older helio
2025-01-07 15:05:50 +02:00
Kostas Kyrimis
e8f43d6858
fix: ExtentTree use of invalidated iterator (#4416)
Fixes accessing and invalidated iterator within ExtendTree
2025-01-07 14:38:32 +02:00
Borys
aaf33d5a1f
test: add test migration in cache mode (#4413) 2025-01-07 13:15:59 +02:00
Roman Gershman
a520ce0195
fix: invalid iterator case in Qlist::Erase (#4414)
The problem was with reverse iterator that was not set properly
when the last node is deleted.

Also, move PushSentinel code into Push.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-07 12:56:23 +02:00
Roman Gershman
a4d243b96f
chore: Introduce small buffer in redis parser (#4076)
* chore: Introduce small buffer in redis parser

This is needed in order to eliminate cases where we return INPUT_PENDING but do not consume the whole string by rejecting just several bytes.
This should simplify buffer management for the caller, so that if they pass a string that
did not result in complete parsed request, at least the whole string is consumed and can be discarded.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

* chore: comments fixes

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-07 11:46:14 +02:00
Shahar Mike
6d03afaa76
chore: Increase migration wait time in test_cluster_migration_huge_container (#4409) 2025-01-07 11:34:43 +02:00
Borys
ed1436bb29
test: move ReplicaofRejectOnLoad test from pytest into unit tests (#4410) 2025-01-07 10:20:36 +02:00
Roman Gershman
21fcf5808e
feat: allow querying of json objects stored as strings (#4399)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-06 21:20:52 +02:00
adiholden
1c0f22f5fa
chore(pytest): add check for rss grow in replicaiton big values (#4406)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-06 08:39:32 +02:00
Shahar Mike
7860a169d9
feat: Yield inside huge values migration serialization (#4197)
* feat: Yield inside huge values migration serialization

With #4144 we break huge values slot migration into multiple commands.
This PR now adds yield between those commands.
It also adds a test that checks that modifying huge values while doing a
migration works well, and that RSS doesn't grow too much.

Fixes #4100
2025-01-05 14:28:45 +00:00
Roman Gershman
ff4add0c9e
chore: deprecate unneeded runtime flags (#4405)
Deprecate multi_exec_mode and track_exec_frequencies.
Remove duplicated call to EnterLameDuck (done also in Service::Shutdown()).

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-05 13:31:59 +00:00
Borys
f663f8e841
fix: provide resp3 option to CapturingReplyBuilder (#4400) 2025-01-05 14:16:02 +02:00
adiholden
0f5eb33e6a
chore(pytest): add timeout per test (#4404)
regression action add 5 minutes timeout per test

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-05 13:19:13 +02:00
adiholden
92c3749c8c
fix(tests): check cluster big snapshot in unit test (#4403)
fix tests: check cluster big snapshot in unit test

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-05 13:18:13 +02:00
Roman Gershman
4f09fe036c
feat: support deletions with meta protocol (#4398)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-03 13:06:24 +00:00
Roman Gershman
6e9409c65c
fix: properly clear tiered stashes upon expirations (#4395)
Fixes #4387

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-02 13:49:13 +00:00
Borys
0832d23f13
fix: allow cluster node load snapshot bigger than maxmemory (#4394) 2025-01-02 13:50:21 +02:00
adiholden
a3ef239ac7
feat(server): refactor allow preempt on journal record (#4393)
* feat server: refactor allow preempt on journal record

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-02 12:16:21 +02:00
Roman Gershman
7a68528022
chore: minor refactorings around dense_set deletions (#4390)
chore: refactorings around deletions

Done as a preparation to introduce asynchronous deletions for sets/zsets/hmaps.
1. Restrict the interface around DbSlice::Del. Now it requires for the iterator to be valid and the checks should
be explicit before the call. Most callers already provides a valid iterator.

2. Some minor refactoring in compact_object_test.
3. Expose DenseSet::ClearStep to allow iterative deletions of elements.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-02 11:35:55 +02:00
adiholden
3b082e42b8
fix(replication): do not log to journal on callback fail (#4392)
fix replication: do not log to journal on callback fail

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-02 09:39:31 +02:00
guyzilla
413ec0a1cf
feat:Adding support for ZMPOP command (#4385)
Signed-off-by: Guy Flysher <guyzila@gmail.com>
2025-01-01 11:34:53 +02:00
Borys
c5b3584dfd
refactor: slot_set don't use stack memory anymore (#4386) 2025-01-01 10:51:38 +02:00
adiholden
810af83074
fix(server): debug populate consume less memory (#4384)
* fix server: debug populate consume less memory

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-01 10:49:17 +02:00
Roman Gershman
2abe2c0ac2
fix: ExternalAllocator::Free with large sizes (#4388)
ExternalAllocator allocates large sizes directly from the extent tree bypassing segment data structure.
Unfortunately, we forgot to align Free() the same way. This PR:

1. Make sure that we add back the allocated range to the extent tree in Free.
2. Rewrite and simplify ExtentTree::Add implementation.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-12-31 18:41:14 +02:00
Stepan Bagritsevich
5e7acbb396
fix(snapshot_test): Fix test_big_value_serialization_memory_limit after adding streams support (#4383)
* fix(snapshot_test): Fix test_big_value_serialization_memory_limit after adding streams support

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* feat: Temporary run test 30 times

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* fix: Remove repeat

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* Revert "fix: Remove repeat"

This reverts commit b7f1ed90b2.

* Revert "feat: Temporary run test 30 times"

This reverts commit 78bfb0fc26.

* Revert "fix(snapshot_test): Fix test_big_value_serialization_memory_limit after adding streams support"

This reverts commit f33036db2a.

* refactor: address comments

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

---------

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2024-12-30 17:38:57 +04:00
Borys
b7532538fe
test: fix test_network_disconnect_during_migration (#4378) 2024-12-30 14:02:12 +02:00
adiholden
22994cf3b7
chore(server): cleanup replication shard sync execution (#4374)
chore server: cleanup replication shard sync execution

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-30 11:52:58 +02:00
Borys
c4be62edc8
fix: return value for DflyMigrateAck (#4379) 2024-12-29 09:37:08 +02:00
Borys
5b9c7e415a
fix: stream memory counting during snapshot loading (#4346)
* fix: stream memory counting during snapshot loading
2024-12-27 09:02:47 +02:00
Stepan Bagritsevich
9fbb301988
chore(snapshot): Small cleanup in Snapshot code (#4377)
* chore(snapshot): Small cleanup in Snapshot code

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* refactor: address comments

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

---------

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2024-12-26 16:02:20 +00:00
Stepan Bagritsevich
0065c27f27
feat(rdb_saver): Support big value serialization for stream (#4376)
fixes dragonflydb#4317

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2024-12-26 15:15:35 +00:00
Stepan Bagritsevich
fb8234ce43
chore(json_family): Remove redundant JSON value copying (#4375)
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2024-12-26 19:00:10 +04:00
adiholden
14fda354cc
fix(pytest): fix failing test test_replication_timeout_on_full_sync (#4370)
fix: increase timeout
fix: remove skip

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-25 14:23:33 +02:00
Roman Gershman
966a1a46fd
feat: support memcache meta responses (#4366)
Fixes #4348 and #3071

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-12-25 07:46:50 +00:00