Commit graph

70 commits

Author SHA1 Message Date
Borys
43c83d29fa
feat: cluster migrations restarts immediately if timeout happens (#4081)
* feat: cluster migrations restarts immediately if timeout happens

* feat: add DEBUG MIGRATION PAUSE command
2024-11-25 16:02:22 +02:00
Andy Dunstall
e05363995f
feat(server): add eval_ro and evalsha_ro (#4091) 2024-11-24 11:53:06 +00:00
Roman Gershman
50473b56aa
chore: decouple reply_builder from ConnectionContext (#4069)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-11-10 21:14:24 +02:00
Roman Gershman
6f6897cef1
chore: pass RedisReplyBuilder explicitly from dragonfly connection (#4009)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-10-29 14:52:09 +02:00
Roman Gershman
408a8a33f2
chore: pass SinkReplyBuilder and Transaction explicitly. Part9 (#3996) 2024-10-25 16:51:37 +03:00
Roman Gershman
4aa0ca1ef7
chore: get rid of MutableSlice (#3952)
* chore: get rid of MutableSlice

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

* chore: comments

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-10-23 21:50:39 +03:00
Roman Gershman
84e22aa658
chore: remove ToUpper calls in main_service (#3947)
* chore: remove ToUpper calls in main_service

Also, test for IsPaused() first to avoid doing more checks for common-case.

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-10-18 14:04:47 +00:00
Kostas Kyrimis
41f7b611d0
chore: enable -Werror=thread-safety and add missing annotations (part 2/2) (#3595)
* add missing annotations
* small mutex fixes
* enable -Werror=thread-safety for clang builds

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-08-30 15:42:30 +03:00
Kostas Kyrimis
839b1be82d
chore: add -Wthread-analysis and annotate (part 1/2) (#3502)
* enable -Wthread-analysis
* add missing annotations
* small fixes

---------

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-08-26 18:22:38 +03:00
Roman Gershman
c9ed3f7b2b
chore: retire TEST_EnableHeartBeat (#3435)
Now unit tests will run the same Hearbeat fiber like in prod.
The whole feature was redundant, with just few explicit settings of maxmemory_limit
I succeeeded to make all unit tests pass.

In addition, this change allows passing a global handler that is called by heartbeat from a single thread.
This is not used yet - preparation for the next PR to break hung up replication connections on a master.

Finally, this change has some non-functional clean-ups and warning fixes to improve code quality.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-08-03 20:17:23 +03:00
Shahar Mike
18ca61d29b
feat(namespaces): Initial support for multi-tenant (#3260)
* feat(namespaces): Initial support for multi-tenant #3050

This PR introduces a way to create multiple, separate and isolated
namespaces in Dragonfly. Each user can be associated with a single
namespace, and will not be able to interact with other namespaces.

This is still experimental, and lacks some important features, such as:
* Replication and RDB saving completely ignores non-default namespaces
* Defrag and statistics either use the default namespace or all
  namespaces without separation

To associate a user with a namespace, use the `ACL` command with the
`TENANT:<namespace>` flag:

```
ACL SETUSER user TENANT:namespace1 ON >user_pass +@all ~*
```

For more examples and up to date info check
`tests/dragonfly/acl_family_test.py` - specifically the
`test_namespaces` function.
2024-07-16 19:34:49 +03:00
Kostas Kyrimis
e1b03d605c
refactor: acl helpers and global tables (#3309)
* remove helper.h and helper.cc and move them to AclFamily class
* remove global tables from acl_commands_def and move them to AclFamily

Signed-off-by: kostas <kostas@dragonflydb.io>
2024-07-15 15:41:13 +03:00
Vladislav
df598e4825
chore: Log db_index in traffic logger (#2951)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2024-04-24 15:13:53 +03:00
Borys
2230397a12
refactor: add cluster namespace (#2948)
* refactor: add cluster namespace, remove extra includes
2024-04-22 21:45:43 +03:00
adiholden
2ad7439128
feat(server): support cluster replication (#2748)
* feat(server): support cluster replication

Signed-off-by: adi_holden <adi@dragonflydb.io>
2024-03-26 15:26:19 +02:00
Roman Gershman
7e0536fd4c
chore: remove core/fibers (#2723)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-03-14 12:02:33 +00:00
Roman Gershman
0e72846457
chore: consolidate facade stats under a single struct (#2368)
* chore: consolidate facade stats under a single struct

Remove connection stats from server state and move them under FacadeStats.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

* chore: fixing comments

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2024-01-03 08:09:00 +00:00
Roman Gershman
bbe3d9303b
feat: introduce transaction statistics in the info output (#2328)
1. How many transactions we processed by type
2. How many transactions we processed by width (number of unique shards).

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-12-23 13:18:49 +02:00
s-shiraki
bd3e57d262
feat(server): Implement NUMSUB subcommand (#2282)
* feat(server): Implement NUMSUB subcommand

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix: test

* fix: build error
2023-12-16 20:42:15 +02:00
Kostas Kyrimis
2703d4635d
feat(acl): add validation for acl keys (#2272)
* add validation for acl keys
* add tests
2023-12-08 17:28:53 +02:00
Vladislav
11ef6623dc
feat: DispatchTracker to replace everything (#2179)
* feat: DispatchTracker

Use a DispatchTracker to track ongoing dispatches for commands that change global state

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-12-05 11:02:11 +03:00
Borys
e6f3522d59
fix: forbid parallel save operations (#2172)
* fix: forbid parallel save operations

* feat: add SAVE option to takeover command
2023-11-21 13:56:27 +02:00
Roy Jacobson
c3a2da559e
feat(server): Implement CLIENT PAUSE (#1875)
* feat(server): Implement CLIENT PAUSE

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-11-15 08:56:49 +02:00
Vladislav
46292968ad
fix(search): Fix replication (#2159)
* fix(search): Support replication

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-11-13 11:58:54 +03:00
Kostas Kyrimis
945d3a39e4
chore(Http): change authed username to default and skip auth on metrics (#1950)
* Update Http auth with username default instead of user
* skip auth for /metrics page
* add/improve tests
* fix a bug with admin port requiring auth on http even if nopass was set
* update helio ref
* update listener class to contain its respective Role
* fix http init to only include admin and main listener
2023-10-03 10:45:37 +03:00
Shahar Mike
2091f53777
opt(lua): Avoid separate hops for lock & unlock on single shard execution (#1900) 2023-09-22 14:09:17 +03:00
Kostas Kyrimis
bbd4c6b636
feat(AclFamily): add acl commands (#1844) 2023-09-15 14:28:36 +03:00
Kostas Kyrimis
866b9afe58
chore: remove locks from acl validation in the hot path (#1765)
* remove locking the registry when Validating users
* deep copies the acl categories in the connection context
* streamline updates to the acl categories via the proactor pool
2023-08-31 21:59:42 +03:00
Kostas Kyrimis
7c43cbf2b5
feat: add validation of acl users (#1743)
* add validation for categories
* add tests
2023-08-29 18:52:46 +03:00
Kostas Kyrimis
898061d738
feat(AclFamilly): add acl list command (#1722)
* Add acl-family source and header
* Add `ACL LIST` command
* Add a simple test to check the default user
2023-08-22 15:33:14 +00:00
Vladislav
4fbd0e38dd
feat: Pipeline squashing (#1619)
* feat: Pipeline squashing

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Signed-off-by: Vladislav <vladislav.oleshko@gmail.com>
Co-authored-by: Kostas Kyrimis <kostaskyrim@gmail.com>
2023-08-17 16:06:48 +03:00
Roman Gershman
a743d7577a
chore: get rid of cmdstats_map (#1687)
cmdstats_map were on the hotpath and are needed only to update the command stats.
Instead, I introduced the stats withing the CommandId itself that we lookup anyways.
Also, it removes fragile dependency on naked command name char* pointers.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-08-10 20:13:02 +03:00
Roman Gershman
f2ddbe7307
chore: refactor a common code into a function (#1685)
Also, fix unreliable time measurement for http latency metric.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-08-10 05:50:43 +03:00
Vladislav
16e512c60d
feat: Refactor command verification before execution (#1652)
* feat: Refactor verifications

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-08-08 12:36:31 +03:00
Vladislav
3bc1e26050
feat: Use const ConnectionContext in VerifyCommand (#1633)
* feat: Use const ConnectionContext in VerifyCommand

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-08-06 11:02:43 +03:00
Roman Gershman
0fe6e36f25
feat: introduce config set command (#1574)
feat: introduce "config set" command

Solves #1551. In detail, we introduce ConfigRegistry that holds
a curated set of flags that could be changed at run-time.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-07-20 15:55:39 +03:00
Roy Jacobson
6d2fcba168
Refactor replication code (#1507)
* refactor: Split redis I/O logic out of dfly::Replica

* Split DFLY shard replication into a separate class.

* Address comments from CR

* Add comments

* remove dead code

* Add a virtual dtor

* * Address comments by Shahar.
* Fix the redis replication code.

* And now fix the Dragonfly replication
2023-07-17 20:48:19 +02:00
Roy Jacobson
52192e0596
fix(server): Initialize ServerFamily with all listeners. (#1485)
* fix(server): Initialize ServerFamily with all listeners.

- Add a test for CLIENT LIST which is the visible result of this.

* use std move
2023-07-02 10:01:54 +02:00
Kostas Kyrimis
99f3284910
feat(server): Add missing fields to INFO PERSISTENCE command (#1408) (#1438)
* Add loading field
* Add rdb_changes_since_last_save field
2023-06-21 23:35:36 +03:00
Roman Gershman
e8fb0b9580
chore: factor out commandid from the server lib (#1253)
Also, move command registry command logic to main_service
where it should reside.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-05-20 14:09:18 +03:00
Chaka
378a0907b2
refactor(cluster): Move CLUSTER commands to their own methods. (#1230)
* Migrate cluster related commands to cluster_family

* Move MutexGuardedObject to its own file.

* Re-add mistakenly removed test

* Remove very cool and useful MutexGuardedObject :)

* clang-format
2023-05-18 14:38:57 +03:00
adiholden
577472eb22
feat(cluster): check command keys ownership (#1194)
* feat(cluster): check command keys ownership on cluster mode

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-05-10 12:06:52 +03:00
Roman Gershman
418f529b0e
fix: 'xgroup help' should show help message (#1159)
Along the way, performs small cleanups in command handling code.
XGROUP HELP is special because it falls out of Dragonfly command taxonomy design,
where a command name determines where its key is located. All other XGROUP subcommands
expect to see XGROUP <subcmd> <key> and this one obviously does not need any key.
I fix it by working around the issue and introduce a dedicated dummy command for this combination.

Fixes #854.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-04-30 09:53:01 +03:00
Vladislav
74a1454409
Collect errors from async lua calls (#1092)
Error collection from async calls
2023-04-22 09:02:22 +03:00
Vladislav
70cf436c05
Lua script async calls (#1070)
Introduces squashing for scripts and a new `redis.acall` command for async commands
2023-04-12 23:37:25 +03:00
Vladislav
90a606851a
chore: Split DispatchCommand into validation and main part (#1071)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-04-11 15:41:49 +03:00
Roman Gershman
c271e13176
chore: import fiber related primitives under dfly namespace (#1012)
This change removes most mentions of boost::fibers or util::fibers_ext.
Instead it introduces "core/fibers.h" file that incorporates most of
the primitives under dfly namespace. This is done in preparation to
switching from Boost.Fibers to helio native fibers.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-03-30 13:26:59 +03:00
Roman Gershman
12abe0bc12
chore: update helio dependency (#984)
Also remove direct references for boost fibers from the code.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-03-24 18:04:05 +03:00
Vladislav
03fb21a280
feat(server): Replace channel slice with store (#938)
* feat(server): Replace channel slice with store

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-03-20 10:46:08 +02:00
Vladislav
03e99a5d96
EVAL multi modes + non atomic modes (#818)
- Implement multi modes for eval
- Implement non atomic mode
- Enhance tests
2023-02-20 09:43:31 +03:00