Commit graph

3063 commits

Author SHA1 Message Date
Roman Gershman
23af41ca07
fix: several bugs (#4500)
1. Fixes #4466 (we did not check validity of the scan cursor)
   We fix it by quietly returning "end of scan" result, i.e. no error are returned.
2. Add 34b1048274c8e50a0cc587a3ed9c383a82bb78c5 sha to the lua_undeclared_keys_shas list.
3. Pull helio that fixes bugs with WaitQueue and IdleTasks: 2ab4412b78
2025-01-24 13:53:09 +02:00
adiholden
69ef9979f0
skip test_migration_one_after_another on epoll (#4499)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-22 17:06:41 +00:00
adiholden
d6adedb066
fix(cluster): crash in cluster migration (#4495)
fix crash in cluster migration

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-22 14:26:53 +02:00
Roman Gershman
451da72c41
chore: improve test_timeout robustness (#4494)
1. use assert_eventually
2. add more logs
3. unrelated - add a stats event to track timeout shutdowns.
2025-01-22 12:24:29 +00:00
Kostas Kyrimis
4a2f2e3d04
chore: add sanitizers to ci workflow (#4462)
* chore: add sanitizers to ci workflow
---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-22 12:00:10 +02:00
Kostas Kyrimis
d44eec2285
fix: test_acl_cat_commands_multi_exec_squash (#4492)
* add sleep in test

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-22 11:15:25 +02:00
Roman Gershman
4b8fa90a67
feat: add 'testing_time' limit option to dfly_bench (#4487)
* feat: add 'testing_time' limit option to dfly_bench
---------

Signed-off-by: Roman Gershman <romange@gmail.com>
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
2025-01-22 10:26:22 +02:00
Roman Gershman
20bc3188fe
feat: implement connection timeout functionality (#4407)
* feat: implement connection timeout functionality

`timeout` argument shuts down idle connections after the specified time.

Fixes #1677
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-21 13:46:57 +02:00
adiholden
986ef7c0e2
fix (stream): XRANGE incorrectly interprets the end parameter (#4443)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-20 21:07:49 +02:00
Stepan Bagritsevich
5d3e3146d3
feat(json_family): Add json_nesting_depth_limit flag (#4444)
* feat: Add json_nesting_depth_limit flag

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* refactor: address comments

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

---------

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2025-01-20 16:49:04 +00:00
Roman Gershman
99c4ab8abe
feat: add support for noreply option (#4485)
feat: add support for noreply and greeat options

noreply option is applicable for memcached.
great option ensure the tcp connection is indeed established before proceeding with loadtest.
2025-01-20 13:59:47 +00:00
Borys
010166525e
feat: allow finish and start migration in the same config (#4486) 2025-01-20 13:52:38 +00:00
Mykhailo Faraponov
91435bc6af
Fix typo in outgoing_slot_migration.cc (#4484)
Signed-off-by: Mykhailo Faraponov <11322032+moredure@users.noreply.github.com>
2025-01-20 12:38:02 +00:00
Roman Gershman
7d0530547b
chore: remove pubsub semantics from Connection::WeakRef (#4483)
Make it generic and move pubsub related logic into channel_store.
2025-01-20 12:32:17 +00:00
Kostas Kyrimis
85cc443448
chore: run regression tests with epoll (#4426)
* chore: run regression tests with epoll

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-20 14:18:21 +02:00
Roman Gershman
4acafa3d47
chore: OpManagerTest fixes (#4480) 2025-01-20 11:38:26 +00:00
Roman Gershman
d2209d9eea
chore: make per-thread QueueBackpressure objects global (#4482)
Before this PR, QueueBackpressure objects are in fact referenced from other threads in an awkward way via
Connection::WeakRef::EnsureMemoryBudget().

This PR removes the complexities of accessing these objects from foreigh threads.
2025-01-20 12:53:36 +02:00
Kostas Kyrimis
b017cdd1a0
fix: hiredis requires df to report version >7.4 (#4474)
* bump up redis version in info command
* add compatibility test
* bump up py dependencies
* fix warnings and deprecated functions

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-20 10:38:37 +02:00
Shahar Mike
6f3c6e3d57
chore: Fix all clang build warnings (#4475)
* chore: Fix all clang build warnings

Also add `-Werror` to clang build in CI.

Fixes #4449

* all build targets

* fix search test
2025-01-20 10:24:07 +02:00
Kostas Kyrimis
c759eb8ce6
chore: replace deprecated HELIO_STACK_CHECK (#4465)
* remove HELIO_STACK_CHECK
* use FLAGS_fiber_safety_margin

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-20 10:17:24 +02:00
Roman Gershman
ac8e1c80f0
chore: dfly_bench now accepts __data__ and __score__ template names (#4477)
* chore: dfly_bench now accepts __data__ and __score__ template names

---------

Signed-off-by: Roman Gershman <romange@gmail.com>
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
2025-01-20 07:59:03 +00:00
Roman Gershman
a936dfe8a5
chore: add Dash::Prefetch function (#4476)
* chore: add Dash::Prefetch function

It's not being used at this time.

* chore: fixes
2025-01-19 20:07:22 +02:00
Roman Gershman
6265f52bff
feat(dev): allow monitoring a valkey server on localhost (#4467) 2025-01-18 10:46:14 +02:00
Daniel M
6f0072e462
Fix test hypothesis (#4380)
* fix:test-hypothesis

---------

Co-authored-by: Roman Gershman <roman@dragonflydb.io>
2025-01-17 07:59:44 +02:00
Stepan Bagritsevich
99f52642c9
fix(sanitizers): Set stack size to 64 KB during tests for Sanitizers (#4454)
* fix(rdb_test): Fix sanitizers for RdbTest

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* Revert "fix(rdb_test): Fix sanitizers for RdbTest"

This reverts commit 39ae4cf9958c517e70c666caee5a7fdd6beeba0d.

* feat: Add default fiber stack size

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* fix: Increase default stack size

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* fix: Specify default stack size for sanitizers

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* fix: Add SANITIZERS to dfly_test_lib

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

---------

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2025-01-16 10:00:42 +01:00
Borys
0e116b1535
fix: slot calculation during transaction squashing (#4460) 2025-01-15 11:43:52 +00:00
Shahar Mike
5ba608b58d
chore: Add stats print for slot migrations (#4456)
* chore: Add stats print for slot migrations

Fixes #4415
2025-01-15 11:06:09 +00:00
Kostas Kyrimis
0eff6c93f2
fix(ci): sanitizers build (#4457)
* compile with clang
* update container to version 24
* move to x86 github runner ubuntu 24
* use compiler-rt instead of libgcc
* add cmake glue
2025-01-15 12:39:43 +02:00
Roman Gershman
e89c15bc6a
fix: resp inline parsing correctly resets itself (#4458)
With empty inline string, the parser failed to continue parsing correctly the non-inline input.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-15 08:45:22 +02:00
Stepan Bagritsevich
f6441df57a
chore: Remove template Callbacks (#4431)
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2025-01-13 15:33:17 +02:00
Roman Gershman
29cde99ca5
chore: pull latest helio (#4446)
This also pulls the latest abseil library 20240722.0
2025-01-13 15:23:08 +02:00
romange
679df5cd81 chore(helm-chart): update to v1.26.1 2025-01-13 07:18:17 +00:00
Roman Gershman
7a7b671095
fix: reply_builder should properly serialize bulk strings endings. (#4441)
Due to a corner-case bug, reply builder could add \0\0 to the end of bulk strings, instead of
\r\n. The bug slipped our tests because redis-py parser most probably does not validate the ending
as long as everything else is consistent.

This PR:
1. Adds a test that catches the bug
2. Adds a debug check that verifies the destination pointer is consistent with the iovec being used.
3. Fixes the bug.

Fixes #4424
2025-01-12 12:32:27 +02:00
Roman Gershman
e39e68276e
chore: dragonfly connection refactorings (#4434)
1. Move socket read code into a dedicated function.
   Remove std:: prefix in the code.
2. Add an optional iouring bufring registration. Currently not being used and is disabled by default.
2025-01-09 13:55:00 +00:00
Roman Gershman
f291ae27cb
fix: show managed stuff on privileged port (#4433)
* fix: show managed stuff on privileged port

* chore: comments
2025-01-09 13:56:14 +02:00
Roman Gershman
c77e7cc09f
fix: improve error propagation with RESTORE commands (#4428)
* fix: improve error propagation with RESTORE commands

Also, provide better logs if AddOrNew function fails adding a new entry
2025-01-09 10:22:47 +00:00
Roman Gershman
f3426bd559
chore: update jsoncons version to 0.178 (#4368)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-08 20:05:50 +02:00
Shahar Mike
5adb976207
chore: Increase migration wait time in test_cluster_migration_huge_container (#4427) 2025-01-08 11:40:40 +00:00
Borys
933c9f0b1c
refactor: remove transaction lib on cluster code dependency (#4417) 2025-01-08 09:38:13 +00:00
Kostas Kyrimis
cb752d90cd
chore: split zset and geo family tests (#4425)
* split zset and geo family tests

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-08 09:34:31 +00:00
Roman Gershman
0a4008385d
fix: switch to SHUTTING_DOWN state unconditionally (#4408)
* fix: switch to SHUTTING_DOWN state unconditionally

During the shutdown sequence always switch to SHUTTING_DOWN.
Make sure that the rest of the code does not break if it can not switch to the desired
global state + some clean ups around state transitions.

Finally, reduce the amount of data in test_replicaof_reject_on_load
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-08 11:28:36 +02:00
Kostas Kyrimis
739bee5c83
chore: split geo and zset families (#4421)
* split geo family from zset family

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-08 10:46:25 +02:00
Kostas Kyrimis
27dc1a44f6
fix: geosearch and georadius response format (#4420)
* return array of arrays only when WITHCOORD, WITHDIST, WITHHASH statements are used

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-07 18:15:06 +02:00
Borys
2fe29c39fd
chore: reduce number of logs in cluster tests (#4419) 2025-01-07 16:04:49 +00:00
Shahar Mike
ec6ac23ebc
chore: fix build on FreeBSD (#4412)
* chore: build on FreeBSD

* docs

* older helio
2025-01-07 15:05:50 +02:00
Kostas Kyrimis
e8f43d6858
fix: ExtentTree use of invalidated iterator (#4416)
Fixes accessing and invalidated iterator within ExtendTree
2025-01-07 14:38:32 +02:00
Borys
aaf33d5a1f
test: add test migration in cache mode (#4413) 2025-01-07 13:15:59 +02:00
Roman Gershman
a520ce0195
fix: invalid iterator case in Qlist::Erase (#4414)
The problem was with reverse iterator that was not set properly
when the last node is deleted.

Also, move PushSentinel code into Push.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-07 12:56:23 +02:00
Roman Gershman
a4d243b96f
chore: Introduce small buffer in redis parser (#4076)
* chore: Introduce small buffer in redis parser

This is needed in order to eliminate cases where we return INPUT_PENDING but do not consume the whole string by rejecting just several bytes.
This should simplify buffer management for the caller, so that if they pass a string that
did not result in complete parsed request, at least the whole string is consumed and can be discarded.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

* chore: comments fixes

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-07 11:46:14 +02:00
Shahar Mike
6d03afaa76
chore: Increase migration wait time in test_cluster_migration_huge_container (#4409) 2025-01-07 11:34:43 +02:00