Commit graph

693 commits

Author SHA1 Message Date
adiholden
b4ef0a06e2
fix cluster: migration crash fix (#4508)
* fix cluster: migration crash fix

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-27 10:39:11 +02:00
adiholden
69ef9979f0
skip test_migration_one_after_another on epoll (#4499)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-22 17:06:41 +00:00
adiholden
d6adedb066
fix(cluster): crash in cluster migration (#4495)
fix crash in cluster migration

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-22 14:26:53 +02:00
Roman Gershman
451da72c41
chore: improve test_timeout robustness (#4494)
1. use assert_eventually
2. add more logs
3. unrelated - add a stats event to track timeout shutdowns.
2025-01-22 12:24:29 +00:00
Kostas Kyrimis
d44eec2285
fix: test_acl_cat_commands_multi_exec_squash (#4492)
* add sleep in test

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-22 11:15:25 +02:00
Roman Gershman
20bc3188fe
feat: implement connection timeout functionality (#4407)
* feat: implement connection timeout functionality

`timeout` argument shuts down idle connections after the specified time.

Fixes #1677
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-21 13:46:57 +02:00
Borys
010166525e
feat: allow finish and start migration in the same config (#4486) 2025-01-20 13:52:38 +00:00
Kostas Kyrimis
85cc443448
chore: run regression tests with epoll (#4426)
* chore: run regression tests with epoll

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-20 14:18:21 +02:00
Kostas Kyrimis
b017cdd1a0
fix: hiredis requires df to report version >7.4 (#4474)
* bump up redis version in info command
* add compatibility test
* bump up py dependencies
* fix warnings and deprecated functions

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-20 10:38:37 +02:00
Kostas Kyrimis
c759eb8ce6
chore: replace deprecated HELIO_STACK_CHECK (#4465)
* remove HELIO_STACK_CHECK
* use FLAGS_fiber_safety_margin

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-20 10:17:24 +02:00
Daniel M
6f0072e462
Fix test hypothesis (#4380)
* fix:test-hypothesis

---------

Co-authored-by: Roman Gershman <roman@dragonflydb.io>
2025-01-17 07:59:44 +02:00
Shahar Mike
5ba608b58d
chore: Add stats print for slot migrations (#4456)
* chore: Add stats print for slot migrations

Fixes #4415
2025-01-15 11:06:09 +00:00
Roman Gershman
f3426bd559
chore: update jsoncons version to 0.178 (#4368)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-08 20:05:50 +02:00
Shahar Mike
5adb976207
chore: Increase migration wait time in test_cluster_migration_huge_container (#4427) 2025-01-08 11:40:40 +00:00
Roman Gershman
0a4008385d
fix: switch to SHUTTING_DOWN state unconditionally (#4408)
* fix: switch to SHUTTING_DOWN state unconditionally

During the shutdown sequence always switch to SHUTTING_DOWN.
Make sure that the rest of the code does not break if it can not switch to the desired
global state + some clean ups around state transitions.

Finally, reduce the amount of data in test_replicaof_reject_on_load
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-08 11:28:36 +02:00
Borys
2fe29c39fd
chore: reduce number of logs in cluster tests (#4419) 2025-01-07 16:04:49 +00:00
Borys
aaf33d5a1f
test: add test migration in cache mode (#4413) 2025-01-07 13:15:59 +02:00
Shahar Mike
6d03afaa76
chore: Increase migration wait time in test_cluster_migration_huge_container (#4409) 2025-01-07 11:34:43 +02:00
adiholden
1c0f22f5fa
chore(pytest): add check for rss grow in replicaiton big values (#4406)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-06 08:39:32 +02:00
Shahar Mike
7860a169d9
feat: Yield inside huge values migration serialization (#4197)
* feat: Yield inside huge values migration serialization

With #4144 we break huge values slot migration into multiple commands.
This PR now adds yield between those commands.
It also adds a test that checks that modifying huge values while doing a
migration works well, and that RSS doesn't grow too much.

Fixes #4100
2025-01-05 14:28:45 +00:00
adiholden
92c3749c8c
fix(tests): check cluster big snapshot in unit test (#4403)
fix tests: check cluster big snapshot in unit test

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-05 13:18:13 +02:00
Roman Gershman
4f09fe036c
feat: support deletions with meta protocol (#4398)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-03 13:06:24 +00:00
Borys
0832d23f13
fix: allow cluster node load snapshot bigger than maxmemory (#4394) 2025-01-02 13:50:21 +02:00
adiholden
810af83074
fix(server): debug populate consume less memory (#4384)
* fix server: debug populate consume less memory

Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-01 10:49:17 +02:00
Stepan Bagritsevich
5e7acbb396
fix(snapshot_test): Fix test_big_value_serialization_memory_limit after adding streams support (#4383)
* fix(snapshot_test): Fix test_big_value_serialization_memory_limit after adding streams support

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* feat: Temporary run test 30 times

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* fix: Remove repeat

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

* Revert "fix: Remove repeat"

This reverts commit b7f1ed90b2.

* Revert "feat: Temporary run test 30 times"

This reverts commit 78bfb0fc26.

* Revert "fix(snapshot_test): Fix test_big_value_serialization_memory_limit after adding streams support"

This reverts commit f33036db2a.

* refactor: address comments

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>

---------

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2024-12-30 17:38:57 +04:00
Borys
b7532538fe
test: fix test_network_disconnect_during_migration (#4378) 2024-12-30 14:02:12 +02:00
Borys
5b9c7e415a
fix: stream memory counting during snapshot loading (#4346)
* fix: stream memory counting during snapshot loading
2024-12-27 09:02:47 +02:00
Stepan Bagritsevich
0065c27f27
feat(rdb_saver): Support big value serialization for stream (#4376)
fixes dragonflydb#4317

Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2024-12-26 15:15:35 +00:00
adiholden
14fda354cc
fix(pytest): fix failing test test_replication_timeout_on_full_sync (#4370)
fix: increase timeout
fix: remove skip

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-25 14:23:33 +02:00
Roman Gershman
966a1a46fd
feat: support memcache meta responses (#4366)
Fixes #4348 and #3071

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-12-25 07:46:50 +00:00
adiholden
937f1bc058
chore(test): disable failing test untill fixed (#4367)
chore test: disable failing test untill fixed

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-25 05:27:00 +02:00
Roman Gershman
a27cce81b1
feat: add support for meta memcache commands (#4362)
This is a stripped down version of supporting the memcache meta requests.

a. Not all meta flags are supported, but TTL, flags, arithmetics are supported.
b. does not include reply support.
c. does not include new semantics that are not part of the older, ascii protocol.

The parser interface has not changed significantly, and the meta commands are emulated
using the old, high level commands like ADD,REPLACE, INCR etc.

See https://raw.githubusercontent.com/memcached/memcached/refs/heads/master/doc/protocol.txt for more details
regarding the meta commands spec.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-12-24 15:33:24 +02:00
Borys
dc81594cd5
test: skip test_network_disconnect_during_migration (#4359) 2024-12-24 10:21:03 +02:00
Borys
76000b9672
fix: test_network_disconnect_during_migration (#4345) 2024-12-22 13:02:53 +02:00
adiholden
e462fc0401
fix(server): use compression for non big values (#4331)
* fix server: use compression for non big values
---------

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-18 22:03:45 +02:00
adiholden
3f68028c08
fix(pytest): call stop for all instances even if stop raise exception (#4339)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-18 18:03:37 +02:00
Borys
6a7931985b
fix: cluster tests stability (#4338) 2024-12-18 13:43:45 +00:00
Shahar Mike
32d71071ae
chore: Disable failing test (#4337) 2024-12-18 12:54:33 +00:00
adiholden
e46717248d
fix (regression tests): skip flaky test (#4336)
skip flaky test

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-18 10:35:47 +00:00
Borys
d0d720a375
test: skip test_network_disconnect_during_migration because it is uns… (#4334)
test: skip test_network_disconnect_during_migration because it is unstable
2024-12-18 08:38:03 +00:00
Roman Gershman
bf410b6e0b
chore: add ability to track connections stuck at send (#4330)
* chore: add ability to track connections stuck at send

Add send_delay_seconds/send_delay_ms metrics.

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Signed-off-by: Roman Gershman <romange@gmail.com>
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
2024-12-18 08:56:36 +02:00
Roman Gershman
19164badf9
fix: potential OOM when first request sent in small bits (#4325)
Before: if socket data arrived in small bits, then CheckForHttpProto would grow
io_buf_ capacity exponentially with each iteration. For example, test_match_http test
easily causes OOM.

This PR ensures that there is always a buffer available - but it grows linearly with the input size.
Currently, the total input in CheckForHttpProto is limited to 1024.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-12-17 13:26:33 +02:00
adiholden
0fe5e86a1a
fix(test): seeder test remove check (#4320)
fix test: seeder test remove check

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-16 16:12:49 +02:00
Borys
8237d8fa81
refactor: remove serialization_max_chunk_size for cluster tests (#4316) 2024-12-16 13:42:56 +02:00
Shahar Mike
af04b558db
fix: Remove hardcoded @assert_eventually 100 times retry (#4318)
This was a subtle and minor bug. Nice catch Borys!
2024-12-16 10:40:45 +00:00
adiholden
027eff2ad3
server: report redis version 7.2.0 to support Sidekiq (#4286)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-16 11:06:08 +02:00
Borys
5e2d0d32b8
test: update logs and test for debug purpose (#4309)
test: update logs and test for devug purpose
2024-12-16 10:25:23 +02:00
Roman Gershman
53637790e8
fix: circular dependency in qlist (#4302)
* fix: circular dependency in qlist

fixes #4294

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

* chore: fixes

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-12-13 11:04:14 +02:00
Kostas Kyrimis
b37287bf14
chore: test metrics for huge value serialization (#4262)
* fix seeder bugs
* add test
* add assertions for huge value metrics

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-12-12 14:19:14 +02:00
Borys
f892d9b7fb
fix: increase cluster migration default timeout (#4293) 2024-12-11 14:39:41 +00:00