Commit graph

3054 commits

Author SHA1 Message Date
Stepan Bagritsevich
5d3e3146d3
feat(json_family): Add json_nesting_depth_limit flag (#4444)
* feat: Add json_nesting_depth_limit flag

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* refactor: address comments

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

---------

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2025-01-20 16:49:04 +00:00
Roman Gershman
99c4ab8abe
feat: add support for noreply option (#4485)
feat: add support for noreply and greeat options

noreply option is applicable for memcached.
great option ensure the tcp connection is indeed established before proceeding with loadtest.
2025-01-20 13:59:47 +00:00
Borys
010166525e
feat: allow finish and start migration in the same config (#4486) 2025-01-20 13:52:38 +00:00
Mykhailo Faraponov
91435bc6af
Fix typo in outgoing_slot_migration.cc (#4484)
Signed-off-by: Mykhailo Faraponov <11322032+moredure@users.noreply.github.com>
2025-01-20 12:38:02 +00:00
Roman Gershman
7d0530547b
chore: remove pubsub semantics from Connection::WeakRef (#4483)
Make it generic and move pubsub related logic into channel_store.
2025-01-20 12:32:17 +00:00
Kostas Kyrimis
85cc443448
chore: run regression tests with epoll (#4426)
* chore: run regression tests with epoll

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-20 14:18:21 +02:00
Roman Gershman
4acafa3d47
chore: OpManagerTest fixes (#4480) 2025-01-20 11:38:26 +00:00
Roman Gershman
d2209d9eea
chore: make per-thread QueueBackpressure objects global (#4482)
Before this PR, QueueBackpressure objects are in fact referenced from other threads in an awkward way via
Connection::WeakRef::EnsureMemoryBudget().

This PR removes the complexities of accessing these objects from foreigh threads.
2025-01-20 12:53:36 +02:00
Kostas Kyrimis
b017cdd1a0
fix: hiredis requires df to report version >7.4 (#4474)
* bump up redis version in info command
* add compatibility test
* bump up py dependencies
* fix warnings and deprecated functions

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-20 10:38:37 +02:00
Shahar Mike
6f3c6e3d57
chore: Fix all clang build warnings (#4475)
* chore: Fix all clang build warnings

Also add `-Werror` to clang build in CI.

Fixes #4449

* all build targets

* fix search test
2025-01-20 10:24:07 +02:00
Kostas Kyrimis
c759eb8ce6
chore: replace deprecated HELIO_STACK_CHECK (#4465)
* remove HELIO_STACK_CHECK
* use FLAGS_fiber_safety_margin

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-20 10:17:24 +02:00
Roman Gershman
ac8e1c80f0
chore: dfly_bench now accepts __data__ and __score__ template names (#4477)
* chore: dfly_bench now accepts __data__ and __score__ template names

---------

Signed-off-by: Roman Gershman <romange@gmail.com>
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
2025-01-20 07:59:03 +00:00
Roman Gershman
a936dfe8a5
chore: add Dash::Prefetch function (#4476)
* chore: add Dash::Prefetch function

It's not being used at this time.

* chore: fixes
2025-01-19 20:07:22 +02:00
Roman Gershman
6265f52bff
feat(dev): allow monitoring a valkey server on localhost (#4467) 2025-01-18 10:46:14 +02:00
Daniel M
6f0072e462
Fix test hypothesis (#4380)
* fix:test-hypothesis

---------

Co-authored-by: Roman Gershman <roman@dragonflydb.io>
2025-01-17 07:59:44 +02:00
Stepan Bagritsevich
99f52642c9
fix(sanitizers): Set stack size to 64 KB during tests for Sanitizers (#4454)
* fix(rdb_test): Fix sanitizers for RdbTest

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* Revert "fix(rdb_test): Fix sanitizers for RdbTest"

This reverts commit 39ae4cf9958c517e70c666caee5a7fdd6beeba0d.

* feat: Add default fiber stack size

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* fix: Increase default stack size

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* fix: Specify default stack size for sanitizers

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* fix: Add SANITIZERS to dfly_test_lib

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

---------

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2025-01-16 10:00:42 +01:00
Borys
0e116b1535
fix: slot calculation during transaction squashing (#4460) 2025-01-15 11:43:52 +00:00
Shahar Mike
5ba608b58d
chore: Add stats print for slot migrations (#4456)
* chore: Add stats print for slot migrations

Fixes #4415
2025-01-15 11:06:09 +00:00
Kostas Kyrimis
0eff6c93f2
fix(ci): sanitizers build (#4457)
* compile with clang
* update container to version 24
* move to x86 github runner ubuntu 24
* use compiler-rt instead of libgcc
* add cmake glue
2025-01-15 12:39:43 +02:00
Roman Gershman
e89c15bc6a
fix: resp inline parsing correctly resets itself (#4458)
With empty inline string, the parser failed to continue parsing correctly the non-inline input.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-15 08:45:22 +02:00
Stepan Bagritsevich
f6441df57a
chore: Remove template Callbacks (#4431)
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2025-01-13 15:33:17 +02:00
Roman Gershman
29cde99ca5
chore: pull latest helio (#4446)
This also pulls the latest abseil library 20240722.0
2025-01-13 15:23:08 +02:00
romange
679df5cd81 chore(helm-chart): update to v1.26.1 2025-01-13 07:18:17 +00:00
Roman Gershman
7a7b671095
fix: reply_builder should properly serialize bulk strings endings. (#4441)
Due to a corner-case bug, reply builder could add \0\0 to the end of bulk strings, instead of
\r\n. The bug slipped our tests because redis-py parser most probably does not validate the ending
as long as everything else is consistent.

This PR:
1. Adds a test that catches the bug
2. Adds a debug check that verifies the destination pointer is consistent with the iovec being used.
3. Fixes the bug.

Fixes #4424
2025-01-12 12:32:27 +02:00
Roman Gershman
e39e68276e
chore: dragonfly connection refactorings (#4434)
1. Move socket read code into a dedicated function.
   Remove std:: prefix in the code.
2. Add an optional iouring bufring registration. Currently not being used and is disabled by default.
2025-01-09 13:55:00 +00:00
Roman Gershman
f291ae27cb
fix: show managed stuff on privileged port (#4433)
* fix: show managed stuff on privileged port

* chore: comments
2025-01-09 13:56:14 +02:00
Roman Gershman
c77e7cc09f
fix: improve error propagation with RESTORE commands (#4428)
* fix: improve error propagation with RESTORE commands

Also, provide better logs if AddOrNew function fails adding a new entry
2025-01-09 10:22:47 +00:00
Roman Gershman
f3426bd559
chore: update jsoncons version to 0.178 (#4368)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-08 20:05:50 +02:00
Shahar Mike
5adb976207
chore: Increase migration wait time in test_cluster_migration_huge_container (#4427) 2025-01-08 11:40:40 +00:00
Borys
933c9f0b1c
refactor: remove transaction lib on cluster code dependency (#4417) 2025-01-08 09:38:13 +00:00
Kostas Kyrimis
cb752d90cd
chore: split zset and geo family tests (#4425)
* split zset and geo family tests

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-08 09:34:31 +00:00
Roman Gershman
0a4008385d
fix: switch to SHUTTING_DOWN state unconditionally (#4408)
* fix: switch to SHUTTING_DOWN state unconditionally

During the shutdown sequence always switch to SHUTTING_DOWN.
Make sure that the rest of the code does not break if it can not switch to the desired
global state + some clean ups around state transitions.

Finally, reduce the amount of data in test_replicaof_reject_on_load
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-08 11:28:36 +02:00
Kostas Kyrimis
739bee5c83
chore: split geo and zset families (#4421)
* split geo family from zset family

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-08 10:46:25 +02:00
Kostas Kyrimis
27dc1a44f6
fix: geosearch and georadius response format (#4420)
* return array of arrays only when WITHCOORD, WITHDIST, WITHHASH statements are used

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-07 18:15:06 +02:00
Borys
2fe29c39fd
chore: reduce number of logs in cluster tests (#4419) 2025-01-07 16:04:49 +00:00
Shahar Mike
ec6ac23ebc
chore: fix build on FreeBSD (#4412)
* chore: build on FreeBSD

* docs

* older helio
2025-01-07 15:05:50 +02:00
Kostas Kyrimis
e8f43d6858
fix: ExtentTree use of invalidated iterator (#4416)
Fixes accessing and invalidated iterator within ExtendTree
2025-01-07 14:38:32 +02:00
Borys
aaf33d5a1f
test: add test migration in cache mode (#4413) 2025-01-07 13:15:59 +02:00
Roman Gershman
a520ce0195
fix: invalid iterator case in Qlist::Erase (#4414)
The problem was with reverse iterator that was not set properly
when the last node is deleted.

Also, move PushSentinel code into Push.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-07 12:56:23 +02:00
Roman Gershman
a4d243b96f
chore: Introduce small buffer in redis parser (#4076)
* chore: Introduce small buffer in redis parser

This is needed in order to eliminate cases where we return INPUT_PENDING but do not consume the whole string by rejecting just several bytes.
This should simplify buffer management for the caller, so that if they pass a string that
did not result in complete parsed request, at least the whole string is consumed and can be discarded.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

* chore: comments fixes

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-07 11:46:14 +02:00
Shahar Mike
6d03afaa76
chore: Increase migration wait time in test_cluster_migration_huge_container (#4409) 2025-01-07 11:34:43 +02:00
Borys
ed1436bb29
test: move ReplicaofRejectOnLoad test from pytest into unit tests (#4410) 2025-01-07 10:20:36 +02:00
Roman Gershman
21fcf5808e
feat: allow querying of json objects stored as strings (#4399)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-06 21:20:52 +02:00
adiholden
1c0f22f5fa
chore(pytest): add check for rss grow in replicaiton big values (#4406)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-06 08:39:32 +02:00
Shahar Mike
7860a169d9
feat: Yield inside huge values migration serialization (#4197)
* feat: Yield inside huge values migration serialization

With #4144 we break huge values slot migration into multiple commands.
This PR now adds yield between those commands.
It also adds a test that checks that modifying huge values while doing a
migration works well, and that RSS doesn't grow too much.

Fixes #4100
2025-01-05 14:28:45 +00:00
Roman Gershman
ff4add0c9e
chore: deprecate unneeded runtime flags (#4405)
Deprecate multi_exec_mode and track_exec_frequencies.
Remove duplicated call to EnterLameDuck (done also in Service::Shutdown()).

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-05 13:31:59 +00:00
Borys
f663f8e841
fix: provide resp3 option to CapturingReplyBuilder (#4400) 2025-01-05 14:16:02 +02:00
adiholden
0f5eb33e6a
chore(pytest): add timeout per test (#4404)
regression action add 5 minutes timeout per test

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-05 13:19:13 +02:00
adiholden
92c3749c8c
fix(tests): check cluster big snapshot in unit test (#4403)
fix tests: check cluster big snapshot in unit test

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-05 13:18:13 +02:00
Roman Gershman
4f09fe036c
feat: support deletions with meta protocol (#4398)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-03 13:06:24 +00:00