Kostas Kyrimis
af5a21bdac
chore: pull helio epoll fix and tests ( #4559 )
...
* pull helio
* re-add epoll test_migration_one_after_another
Signed-off-by: kostas <kostas@dragonflydb.io>
2025-02-06 12:25:54 +02:00
Stepan Bagritsevich
4ba1142cb3
fix(stream_family): Fix journaling in the XADD and XTRIM commands ( #4448 )
...
* fix(stream_family): Fix journaling in the XADD and XTRIM commands
fixes dragonflydb#4202
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
* fix(replication_test): Fix a typo
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
* feat(replication_test): Add test_stream_approximate_trimming test
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
* refactor: address comments
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
* refactor: address comments 2
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
* fix(stream_family): Fix stream id journaling in the XADD command
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
---------
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2025-02-05 14:43:26 +00:00
Shahar Mike
f8667235db
chore: Add FakeRedis capture/compare to more cluster tests and seed during migration ( #4542 )
...
* chore: Add FakeRedis capture/compare to more cluster tests
Note that we can't add such a test to `test_cluster_native_client`
because we don't do a migration there, just re-assign slots, meaning
that data is lost (by desgin?)
Fixes #4429
* Seed during migrations
* Add TODO
2025-02-05 14:34:42 +02:00
Roman Gershman
6d1c22b64c
chore: GlobMatcher uses now reflex::Matcher regex engine ( #4528 )
...
Also consolidate benchmarking low level routines undeer dfly_core_test
```
BM_ParseFastFloat 707 ns 707 ns 4005656
BM_ParseDoubleAbsl 1460 ns 1460 ns 1927158
BM_MatchGlob/1000 121 ns 121 ns 23701780
BM_MatchGlob/10000 512 ns 512 ns 5481405
BM_MatchFindSubstr/1000 123 ns 123 ns 31114255
BM_MatchFindSubstr/10000 1126 ns 1126 ns 2522019
BM_MatchReflexFind/1000 118 ns 118 ns 22442417
BM_MatchReflexFind/10000 512 ns 512 ns 5414329
BM_MatchReflexFindStar/1000 106 ns 106 ns 26276727
BM_MatchReflexFindStar/10000 717 ns 717 ns 3719605
BM_MatchStd/1000 19782 ns 19779 ns 128020
BM_MatchStd/10000 199809 ns 199781 ns 13837
BM_MatchRedisGlob/1000 1601 ns 1601 ns 1754635
BM_MatchRedisGlob/10000 16494 ns 16493 ns 171585
BM_MatchRe2/1000 1039 ns 1039 ns 2709486
BM_MatchRe2/10000 10041 ns 10040 ns 281296
```
What's curious is that now matching `*foobar*` on string is faster than
searching for 'foobar` using string::find() (BM_MatchGlob vs BM_MatchFindSubstr)
Improvement vs Redis is 10-30 times faster (BM_MatchRedisGlob vs BM_MatchGlob).
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-02-05 10:29:51 +02:00
Borys
6c68519f02
fix: test_snapshoting_during_migration ( #4555 )
2025-02-04 13:56:24 +00:00
Roman Gershman
8c937ebf37
chore: clean up of deprecated flags ( #4545 )
...
Also fix sentinel test by using a precise redis-server version.
Finally, add pytest warnings filter to reduce noise
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-02-02 20:03:23 +02:00
adiholden
b4ef0a06e2
fix cluster: migration crash fix ( #4508 )
...
* fix cluster: migration crash fix
Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-27 10:39:11 +02:00
adiholden
69ef9979f0
skip test_migration_one_after_another on epoll ( #4499 )
...
Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-22 17:06:41 +00:00
adiholden
d6adedb066
fix(cluster): crash in cluster migration ( #4495 )
...
fix crash in cluster migration
Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-22 14:26:53 +02:00
Roman Gershman
451da72c41
chore: improve test_timeout robustness ( #4494 )
...
1. use assert_eventually
2. add more logs
3. unrelated - add a stats event to track timeout shutdowns.
2025-01-22 12:24:29 +00:00
Kostas Kyrimis
d44eec2285
fix: test_acl_cat_commands_multi_exec_squash ( #4492 )
...
* add sleep in test
Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-22 11:15:25 +02:00
Roman Gershman
20bc3188fe
feat: implement connection timeout functionality ( #4407 )
...
* feat: implement connection timeout functionality
`timeout` argument shuts down idle connections after the specified time.
Fixes #1677
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-21 13:46:57 +02:00
Borys
010166525e
feat: allow finish and start migration in the same config ( #4486 )
2025-01-20 13:52:38 +00:00
Kostas Kyrimis
85cc443448
chore: run regression tests with epoll ( #4426 )
...
* chore: run regression tests with epoll
Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-20 14:18:21 +02:00
Kostas Kyrimis
b017cdd1a0
fix: hiredis requires df to report version >7.4 ( #4474 )
...
* bump up redis version in info command
* add compatibility test
* bump up py dependencies
* fix warnings and deprecated functions
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-20 10:38:37 +02:00
Kostas Kyrimis
c759eb8ce6
chore: replace deprecated HELIO_STACK_CHECK ( #4465 )
...
* remove HELIO_STACK_CHECK
* use FLAGS_fiber_safety_margin
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
2025-01-20 10:17:24 +02:00
Daniel M
6f0072e462
Fix test hypothesis ( #4380 )
...
* fix:test-hypothesis
---------
Co-authored-by: Roman Gershman <roman@dragonflydb.io>
2025-01-17 07:59:44 +02:00
Shahar Mike
5ba608b58d
chore: Add stats print for slot migrations ( #4456 )
...
* chore: Add stats print for slot migrations
Fixes #4415
2025-01-15 11:06:09 +00:00
Roman Gershman
f3426bd559
chore: update jsoncons version to 0.178 ( #4368 )
...
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-08 20:05:50 +02:00
Shahar Mike
5adb976207
chore: Increase migration wait time in test_cluster_migration_huge_container ( #4427 )
2025-01-08 11:40:40 +00:00
Roman Gershman
0a4008385d
fix: switch to SHUTTING_DOWN state unconditionally ( #4408 )
...
* fix: switch to SHUTTING_DOWN state unconditionally
During the shutdown sequence always switch to SHUTTING_DOWN.
Make sure that the rest of the code does not break if it can not switch to the desired
global state + some clean ups around state transitions.
Finally, reduce the amount of data in test_replicaof_reject_on_load
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-08 11:28:36 +02:00
Borys
2fe29c39fd
chore: reduce number of logs in cluster tests ( #4419 )
2025-01-07 16:04:49 +00:00
Borys
aaf33d5a1f
test: add test migration in cache mode ( #4413 )
2025-01-07 13:15:59 +02:00
Shahar Mike
6d03afaa76
chore: Increase migration wait time in test_cluster_migration_huge_container ( #4409 )
2025-01-07 11:34:43 +02:00
adiholden
1c0f22f5fa
chore(pytest): add check for rss grow in replicaiton big values ( #4406 )
...
Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-06 08:39:32 +02:00
Shahar Mike
7860a169d9
feat: Yield inside huge values migration serialization ( #4197 )
...
* feat: Yield inside huge values migration serialization
With #4144 we break huge values slot migration into multiple commands.
This PR now adds yield between those commands.
It also adds a test that checks that modifying huge values while doing a
migration works well, and that RSS doesn't grow too much.
Fixes #4100
2025-01-05 14:28:45 +00:00
adiholden
92c3749c8c
fix(tests): check cluster big snapshot in unit test ( #4403 )
...
fix tests: check cluster big snapshot in unit test
Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-05 13:18:13 +02:00
Roman Gershman
4f09fe036c
feat: support deletions with meta protocol ( #4398 )
...
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2025-01-03 13:06:24 +00:00
Borys
0832d23f13
fix: allow cluster node load snapshot bigger than maxmemory ( #4394 )
2025-01-02 13:50:21 +02:00
adiholden
810af83074
fix(server): debug populate consume less memory ( #4384 )
...
* fix server: debug populate consume less memory
Signed-off-by: adi_holden <adi@dragonflydb.io>
2025-01-01 10:49:17 +02:00
Stepan Bagritsevich
5e7acbb396
fix(snapshot_test): Fix test_big_value_serialization_memory_limit after adding streams support ( #4383 )
...
* fix(snapshot_test): Fix test_big_value_serialization_memory_limit after adding streams support
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
* feat: Temporary run test 30 times
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
* fix: Remove repeat
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
* Revert "fix: Remove repeat"
This reverts commit b7f1ed90b278bfb0fc26f33036db2a
2024-12-30 17:38:57 +04:00
Borys
b7532538fe
test: fix test_network_disconnect_during_migration ( #4378 )
2024-12-30 14:02:12 +02:00
Borys
5b9c7e415a
fix: stream memory counting during snapshot loading ( #4346 )
...
* fix: stream memory counting during snapshot loading
2024-12-27 09:02:47 +02:00
Stepan Bagritsevich
0065c27f27
feat(rdb_saver): Support big value serialization for stream ( #4376 )
...
fixes dragonflydb#4317
Signed-off-by: Stepan Bagritsevich <stefan@dragonflydb.io>
2024-12-26 15:15:35 +00:00
adiholden
14fda354cc
fix(pytest): fix failing test test_replication_timeout_on_full_sync ( #4370 )
...
fix: increase timeout
fix: remove skip
Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-25 14:23:33 +02:00
Roman Gershman
966a1a46fd
feat: support memcache meta responses ( #4366 )
...
Fixes #4348 and #3071
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-12-25 07:46:50 +00:00
adiholden
937f1bc058
chore(test): disable failing test untill fixed ( #4367 )
...
chore test: disable failing test untill fixed
Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-25 05:27:00 +02:00
Roman Gershman
a27cce81b1
feat: add support for meta memcache commands ( #4362 )
...
This is a stripped down version of supporting the memcache meta requests.
a. Not all meta flags are supported, but TTL, flags, arithmetics are supported.
b. does not include reply support.
c. does not include new semantics that are not part of the older, ascii protocol.
The parser interface has not changed significantly, and the meta commands are emulated
using the old, high level commands like ADD,REPLACE, INCR etc.
See https://raw.githubusercontent.com/memcached/memcached/refs/heads/master/doc/protocol.txt for more details
regarding the meta commands spec.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-12-24 15:33:24 +02:00
Borys
dc81594cd5
test: skip test_network_disconnect_during_migration ( #4359 )
2024-12-24 10:21:03 +02:00
Borys
76000b9672
fix: test_network_disconnect_during_migration ( #4345 )
2024-12-22 13:02:53 +02:00
adiholden
e462fc0401
fix(server): use compression for non big values ( #4331 )
...
* fix server: use compression for non big values
---------
Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-18 22:03:45 +02:00
adiholden
3f68028c08
fix(pytest): call stop for all instances even if stop raise exception ( #4339 )
...
Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-18 18:03:37 +02:00
Borys
6a7931985b
fix: cluster tests stability ( #4338 )
2024-12-18 13:43:45 +00:00
Shahar Mike
32d71071ae
chore: Disable failing test ( #4337 )
2024-12-18 12:54:33 +00:00
adiholden
e46717248d
fix (regression tests): skip flaky test ( #4336 )
...
skip flaky test
Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-18 10:35:47 +00:00
Borys
d0d720a375
test: skip test_network_disconnect_during_migration because it is uns… ( #4334 )
...
test: skip test_network_disconnect_during_migration because it is unstable
2024-12-18 08:38:03 +00:00
Roman Gershman
bf410b6e0b
chore: add ability to track connections stuck at send ( #4330 )
...
* chore: add ability to track connections stuck at send
Add send_delay_seconds/send_delay_ms metrics.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Signed-off-by: Roman Gershman <romange@gmail.com>
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
2024-12-18 08:56:36 +02:00
Roman Gershman
19164badf9
fix: potential OOM when first request sent in small bits ( #4325 )
...
Before: if socket data arrived in small bits, then CheckForHttpProto would grow
io_buf_ capacity exponentially with each iteration. For example, test_match_http test
easily causes OOM.
This PR ensures that there is always a buffer available - but it grows linearly with the input size.
Currently, the total input in CheckForHttpProto is limited to 1024.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-12-17 13:26:33 +02:00
adiholden
0fe5e86a1a
fix(test): seeder test remove check ( #4320 )
...
fix test: seeder test remove check
Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-12-16 16:12:49 +02:00
Borys
8237d8fa81
refactor: remove serialization_max_chunk_size for cluster tests ( #4316 )
2024-12-16 13:42:56 +02:00
