Commit graph

2257 commits

Author SHA1 Message Date
Vladislav
e45c1e92a2
chore: Refactor string span management (#3165)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-06-18 10:37:32 +03:00
Vladislav
6291c04016
chore(search): Add __vector_score alias (#3186) 2024-06-17 23:39:38 +03:00
Roman Gershman
6e0e3cb350
chore: add a lexer test covering KNN query (#3185) 2024-06-17 18:42:30 +03:00
Roman Gershman
e6a9e0dbc2
chore: allow calling Context::ReportError from I/O dispatch fiber (#3182)
Fixes the issue when we are trying to Report the error from JournalStreamer::OnCompletion
that is called from a proactor callback directly from the I/O fiber.

We can not use fiber mutexes in the I/O fiber so we switch to regular mutex and reduce its critical sections
as much as possible.

Should fix the following failure:
2628431732 (step):6:622

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-06-17 10:58:24 +03:00
Kostas Kyrimis
8d82581c96
chore: small acl compat changes (#3177)
* print categories and commands in lower case instead of capital case
* fix a bug of default user inheriting the wrong acl rules on new connections
* move keys position to be after password when printed from an acl command
2024-06-17 09:40:41 +03:00
adiholden
805c024fc2
fix(server): fix bug in replication on cached mode (#3156)
* fix server: fix replication on cached mode

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-06-17 08:37:58 +03:00
Shahar Mike
664cba9e10
fix(cluster): Wait before all access to slot migrations (#3180)
* fix(cluster): Wait before all access to slot migrations

* refactor

* reduce lock scope
2024-06-16 15:10:15 +03:00
Roman Gershman
32844e0f3d
chore: add parsing support for the rest of VECTOR parameters (#3179) 2024-06-16 12:29:27 +03:00
Borys
4f1be8f3dc
feat: add incoming migration error processing (#3175) 2024-06-16 12:18:18 +03:00
Roman Gershman
688f286f46
chore: improve ft.create for vector search (#3178)
metric name in vector parameters should be case insensitive

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-06-16 08:18:16 +03:00
Shahar Mike
93b4193141
fix(cluster): Wait for flow creation before cancelling it (#3159)
* fix(cluster): Wait for flow creation before cancelling it

This subtle bug could appear if a migration fails at exactly the wrong
time:
1. Thread1 creates an outgoing flow and starts the migration
2. Thread1's flow fails for whatever reason
3. Thread1 attempts to cancel all flows
4. Thread2 did *not* yet create its flow
5. Thread1 skips non-existing Thread2 flow
6. Thread2 now creates the flow
7. Upon destruction of entire migration, Thread2's fiber has not been
   joined, crashing Dragonfly

The fix is quite simple though :)

Fixes #3139

* split creation and invocation
2024-06-13 16:49:04 +03:00
Vladislav
c08719117c
feat(json): MSET (#3167)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-06-13 12:33:24 +03:00
Roman Gershman
a80063189e
chore: Streamer is rewritten with async interface (#3108)
* chore: Streamer is rewritten with async interface

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-06-13 12:29:06 +03:00
Kostas Kyrimis
d2ae0ab75c
fix: acl compatibility (#3147)
* remove acl categories from context and all acl checks
* category assign,ent now assigns all the acl commands for that category to the user
* introduce modification order of acl's per user
* acl rules are now printed in the same order as in redis/valkey
* remove old user_registry_test which was part of the poc
2024-06-13 10:56:30 +03:00
Kostas Kyrimis
165631a5aa
fix(ci): add missing docker option on reg tests workflow (#3172)
* add missing option for iouring and docker
2024-06-13 10:55:45 +03:00
Kostas Kyrimis
a9ffb22384
fix(ci): docker not using iouring (#3169)
* fix(ci): docker not using iouring

* fix errors
2024-06-12 16:38:02 +03:00
Kostas Kyrimis
3f8c8176cb
feat: add noloop subcommand in client tracking (#3164)
* add noloop subcommand
2024-06-11 14:15:10 +00:00
Kostas Kyrimis
2089760755
feat: client tracking optout (#3158)
* implement client tracking optout
* fix a bug of allowing client caching no without tracking optout
* fix error message compatibility with redis/valkey
2024-06-11 16:32:38 +03:00
Kostas Kyrimis
7f9a13bb50
fix: lua and client tracking (#3163)
* add missing tracking_cb_ invkoe in RunSquashedMultiCb
* add test
2024-06-11 16:24:44 +03:00
Roman Gershman
007d4854db
chore: Introduce pipeline back-pressure (#3152)
* chore: Introduce pipeline back-pressure

Also, improve synchronization primitives and replace them with
thread-local variations.

Before the change, on my local machine with the dragonfly running with 8 threads,
`memtier_benchmark  -c 10 --threads 8  --command="PING"  --key-maximum 100000000  --hide-histogram --distinct-client-seed --pipeline=20 --test-time=10`

reached 10M qps with 0.327ms p99.9.

After the change, the same command showed 13.8M qps with 0.2ms p99.9
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-06-10 12:39:41 +03:00
Vladislav
8eb9d48c3a
feat(search): basic FT.ALTER (#3148)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-06-08 09:16:29 +03:00
Vladislav
1053633a6a
feat(search): Tag field options (separator, case sensitivity) (#3144)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-06-07 22:34:06 +03:00
Kostas Kyrimis
0edcbc80ef
chore: recommit client tracking (#3136)
* bring back client tracking
* disable all client commands in squashing
* allow all client commands in multi/exec block
2024-06-07 15:34:44 +03:00
Borys
39dd73fc71
fix: fix bug in cluster/slot_set (#3143)
* fix: fix bug in cluster/slot_set

* fix: fix slot flushes
2024-06-07 14:31:11 +03:00
Kostas Kyrimis
3c6d72b93a
chore: update helio (#3150) 2024-06-07 13:21:59 +03:00
Borys
66a524a026
test: skip test_cluster_migration_cancel, it is broken (#3146) 2024-06-06 16:33:08 +03:00
Roman Gershman
6498f88619
chore: a small cleanup (#3140)
* chore: a small cleanup

No functional changes.

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-06-06 15:16:52 +03:00
Shahar Mike
53c52273b4
feat(cluster): Add important logs (#3138)
Issue: #2829
2024-06-06 11:29:55 +03:00
Vladislav
37eea74154
chore: Add option to disable aws dependency (#3077)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-06-06 09:21:00 +03:00
Roman Gershman
ce2e127183
chore: call breaker_cb_ on shutdown (#3128)
* chore: call breaker_cb_ on shutdown
---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-06-06 08:09:40 +03:00
Shahar Mike
229eeeb014
fix: Fix live-lock in connection test (#3135)
fix: Fix livelocking connection test
2024-06-05 21:30:02 +03:00
Roman Gershman
9f09104c61
chore: get rid of kv_args and replace it with slices to full_args (#3101)
* Revert "Revert "chore: get rid of kv_args and replace it with slices to full_… (#3024)"

This reverts commit 25e6930ac3.

Fixing the performance bug caused by applying equality operator
on two spans inside ShardArgs::Iterator::operator==

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-06-05 14:25:26 +03:00
Shahar Mike
1fb250b64f
test(cluster_mgr): Add tests for cluster_mgr.py (#3129) 2024-06-05 12:12:58 +03:00
romange
58024d9e8f chore(helm-chart): update to v1.19.0 2024-06-05 05:50:12 +00:00
Roman Gershman
90b5ec4565
fix: upload action in version release (#3127)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-06-04 15:46:17 +00:00
Vladislav
6a873b4f1c
feat(tiering): Simple snapshotting (#3073)
* feat(tiering): Simple snapshotting

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-06-04 17:15:21 +03:00
Roman Gershman
b02521cf51
chore: prevent Dispatch fiber to be launched during migration (#3123)
* chore: prevent Dispatch fiber to be launched during the connection migration
2024-06-04 14:13:48 +00:00
Shahar Mike
5b731f163c
feat(cluster_mgr): Fix migration action (#3124) 2024-06-04 13:27:42 +03:00
Borys
7606af706f
fix: fix RestoreStreamer to prevent buckets skipping #2830 (#3119)
* fix: fix RestoreStreamer to prevent bucket skipping #2830
2024-06-04 11:50:03 +03:00
Shahar Mike
bcbcc5a2c6
feat(cluster_mgr): Take over command (#3120) 2024-06-04 11:39:08 +03:00
adiholden
2a5a53f1d9
Revert "refactor client tracking, fix atomicity, squashing and multi/… (#3122)
Revert "refactor client tracking, fix atomicity, squashing and multi/exec (#2970)"

This reverts commit b1063f7823.
2024-06-04 10:34:32 +03:00
adiholden
6e33261402
fix(server): fix compatibility with rdb snapshot (#3121)
* fix server: fix compatibility with rdb snapshot


Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-06-04 06:28:18 +00:00
Kostas Kyrimis
b1063f7823
refactor client tracking, fix atomicity, squashing and multi/exec (#2970)
* add partial support for CLIENT CACHING TRUE (only to be used with TRACKING OPTIN)
* add OPTIN to CLIENT TRACKING command
* refactor client tracking to respect transactional atomicity
* fixed multi/exec and disabled squashing with client tracking
* add tests
2024-06-03 22:14:30 +03:00
Shahar Mike
6e6c91aeaf
feat(cluster_mgr): Improvements to cluster_mgr.py (#3118)
Make sure attached node is in right mode
Enable detaching nodes
2024-06-03 19:05:17 +00:00
Kostas Kyrimis
3924fcad68
chore: pull helio add test for tls deadlock (#3111)
* pull helio
* add test that covers tls deadlock
2024-06-03 14:13:47 +00:00
Shahar Mike
4a1831c911
feat(server): Pipeline and dispatch on /metrics (#3117) 2024-06-03 14:11:39 +03:00
Borys
4371fea436
fix: fix cluster_fuzzy_migration test (#3116) 2024-06-02 14:12:52 +03:00
Roman Gershman
0ab1c636f7
chore: minor fixes + remove redundant DCHECK (#3115)
* chore: minor fixes + remove redundant DCHECK

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-06-02 12:22:05 +03:00
Borys
644dc3f139
New test for cluster migration: connection issue (#3102)
* test: update test_config_consistency, 
update test_cluster_data_migration, 
new cluster migration test for network issues
2024-06-02 09:16:03 +03:00
Vladislav
b3ca27068a
chore: zset error log (from 1.18 branch) (#3109) 2024-05-31 16:00:09 +03:00