Commit graph

240 commits

Author SHA1 Message Date
s-shiraki
bd3e57d262
feat(server): Implement NUMSUB subcommand (#2282)
* feat(server): Implement NUMSUB subcommand

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix: test

* fix: build error
2023-12-16 20:42:15 +02:00
Vladislav
e273f7a2b8
fix(server): handle no-key-transactional commands in multi/exec (#2279)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-12-12 19:34:02 +03:00
Yue Li
64bbfc7063
feat(server): Support CLIENT TRACKING subcommand (1/2) (#2277)
The client tracking state is set by CLIENT TRACKING subcommand as well
as upon client disconnection.

Track the keys of a readonly command by maintaining mapping that maps
keys to the sets of tracking clients.
2023-12-08 23:13:55 -08:00
Kostas Kyrimis
2703d4635d
feat(acl): add validation for acl keys (#2272)
* add validation for acl keys
* add tests
2023-12-08 17:28:53 +02:00
Yue Li
2e914efb52
bug(server): remove CO::READONLY for QUIT command. (#2263)
remove readonly for QUIT command.
2023-12-06 18:23:35 +02:00
Vladislav
5f58b3aafa
fix(connection): Add WeakRef to replace pubsub wait token (#2227)
Add WeakRef to connection to allow tracking it

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-12-06 12:07:31 +03:00
Borys
33d0879416
refactor: conn_context and reply_builder refactoring (#2251)
* refactor: conn_context and reply_builder refactoring
2023-12-06 08:23:32 +02:00
Vladislav
11ef6623dc
feat: DispatchTracker to replace everything (#2179)
* feat: DispatchTracker

Use a DispatchTracker to track ongoing dispatches for commands that change global state

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-12-05 11:02:11 +03:00
Vladislav
fd3fcdcbb4
fix(server): Unwatch keys during connection close (#2231) 2023-11-30 11:56:59 +02:00
Roman Gershman
0c5bb7b894
fix: regression test failures (#2226)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-11-27 21:41:52 +02:00
Roman Gershman
b853b2ab00
fix: memcached VERSION is now parseable by php-memcached client (#2220)
The DF version is being unparseable by Memcached::getVersion() that expects n.n.n string.
Change the version to emulate the old memcached server.
The DF version can still be fetched via Memcached::getStats() function.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-11-27 20:54:00 +02:00
Roman Gershman
92614477b7
fix: fix cmd latency units in /metrics (#2217)
Also,
1. rebase helio dependency
2. get rid of varz counters that are superseded by
   commands_total/commands_duration_seconds_total metrics

Resolves #2213

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-11-27 11:44:16 +02:00
Vladislav
d6044edbab
fix(squashing): Reset base command id (#2209)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-11-26 12:40:37 +02:00
Roman Gershman
7d53d196aa
chore: improve performance of mget operation (#2212)
Specifically, allocate only a single blob when returning multiple entries from a shard.
In addition, refactor and unify MGetResponse between string family code and ReplyBuilder code.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-11-24 14:13:07 +02:00
Borys
e6f3522d59
fix: forbid parallel save operations (#2172)
* fix: forbid parallel save operations

* feat: add SAVE option to takeover command
2023-11-21 13:56:27 +02:00
Vladislav
d21f82a5f9
chore: connection fixes (#2192)
* chore: add more states to client connections

* fix: clear pipelined messages before close

* fix: skip same thread on backpressure
---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Roman Gershman <roman@dragonflydb.io>
2023-11-20 17:08:12 +00:00
adiholden
c95f4961be
fix(server): client pause fix on pipeline squash (#2180)
* fix(server): client pause fix on pipeline squash

allow squashing commands on pause
move await on client pause inside InvokeCommand - this way all flows of command invoke will read pause state

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-11-16 13:30:02 +02:00
Roy Jacobson
c3a2da559e
feat(server): Implement CLIENT PAUSE (#1875)
* feat(server): Implement CLIENT PAUSE

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-11-15 08:56:49 +02:00
Vladislav
564e38c05c
chore: lower takeover test load, add comments (#2151)
* chore: lower takeover test load, add comments

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-11-12 12:08:05 +03:00
Roman Gershman
4201ac416e
chore: remove Command step argument (#2150)
It will be represented via INTERLEAVED_KEYS option.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-11-10 19:10:56 +02:00
Vladislav
0da488d99e
chore(eval): Don't crash on unsupported script commands (#2138)
* chore(eval): Don't crash on unsupported script commands

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-11-08 10:21:52 +02:00
Kostas Kyrimis
f68e1ef7e3
fix(memcached): parsing multi key get command (#2122)
* remove limit of 8 keys per command
* refactor (small) of the parsing logic
* add test
2023-11-06 11:27:46 +02:00
Shahar Mike
414f6aad2c
feat(server): Add /clusterz page with info on cluster (#2115)
It's not the prettiest, but it's functional.
2023-11-03 08:09:52 +02:00
Yue Li
00f1e3d578
feat(server): perform eviction upon memory pressure in cache mode (#2084)
* fixes #1936

Eviction Implementation
This patch provides a very simple eviction implementation for the interface mentioned above. In my opinion, the eviction algorithm approximates an LRU policy given that normal buckets always store the most recently accessed data while stash buckets are holding less active data.

The algorithm first selects a small set of segments as eviction targets. Starting from the last slot of the last stash bucket in each of the segments, we walk backward to evict key-value pairs stored in each visited slot. The eviction stopped either when a target memory release goal or the max number of evicted key-value pairs is reached. Therefore, we can upper bound the eviction time through the following two parameters that can be set when DF starts.  Note that these two parameters could be retrieved and changed by user through CONFIG GET and CONFIG SET commands.

---------

Signed-off-by: Yue Li <61070669+theyueli@users.noreply.github.com>
2023-11-01 11:11:27 -07:00
Kostas Kyrimis
84744a4004
fix(ReplyGuard): set member variable on construction and remove unused (#2082)
* set member variable to point to the reply builder
* remove redundant checks
* import missing signal in instance.py
2023-10-30 13:54:27 +02:00
Kostas Kyrimis
1538f66a35
fix(AclFamily): renaming commands and subcommands (#2076)
* fix renaming ACL commands
* fix ACL subcommands to be treated as such. For example: ACL list would print all ACL subcommands. Now it only prints ACL
* add tests
2023-10-29 18:01:37 +02:00
Shahar Mike
502efd80b2
fix(server): Correctly set expiration from Lua scripts (#2080)
We used to set `time_now_ms_` only in the non-squashed execution path.

Fixes #2034
2023-10-27 13:13:47 +03:00
Kostas Kyrimis
b655fc7415
chore(reply_builder): add dcheck that each command invocation has rep… (#2067)
* chore(reply_builder): add dcheck that each command invocation has replied
2023-10-26 19:44:32 +03:00
Shahar Mike
b2e989a4fb
fix(server): Fix crash when connection uses multiple lua scripts (#2073) 2023-10-25 19:44:09 +00:00
Roman Gershman
d9cb7453fb
wrap dragonfly_test with gdb inside a CI (#2050)
chore: run dragonfly_test with epoll under gdb

Also, update helio that provide a stacktrace under musl libc (alpine linux).
This version of helio updates absl version as well.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-10-22 20:05:27 +03:00
Kostas Kyrimis
1d02e12ad1
chore: call debug stacktrace on SIGUSR1 (#2012)
* add macro to install a signal handler that prints the contents of debug stacktrace on SIGUSR1
* add this on regTests
2023-10-20 10:50:55 +03:00
Shahar Mike
bc0064e94d
opt(server): Migrate connections which invoke Lua scripts to their target thread. (#2037)
This is enabled by default, but can be disabled via `--migrate_connections=false`.

Measuring with BullMQ benchmark I see a gain of almost 10% in
throughput. I haven't measured, but it's supposed to also reduce
latency.
2023-10-19 22:31:07 +03:00
Vladislav
22a703de38
feat(server): Support COMMAND INFO (#2035)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-10-17 18:33:55 +03:00
Vladislav
4b387cebe2
fix(server): Exclude eval from pipeline squashing (#2027)
* fix(server): Exclude eval from pipeline squashing

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-10-16 17:48:42 +03:00
Shahar Mike
9e2a9fd5e5
fix(server): Fix shardlocal eval with round-robin prefix (#2024) 2023-10-15 19:36:41 +03:00
Vladislav
826b1d81a3
chore: Enable squashing by default (#2011)
* chore: Enable squashing
2023-10-15 12:17:52 +03:00
Vladislav
cb9a45f2a9
fix(server): Don't recompute shard for squashed stub tx (#2017)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-10-15 09:27:58 +03:00
Roman Gershman
ec87114f66
fix: buffer overrun when passing long command name from lua (#2008)
Also, few additional changes that do not affect functionality.
1. make sure passed arguments to DispatchCommand are `\0` delimited
   during pipelining.
2. extend lua malloc hook to call precise functions - to help with cpu profiling.
3. reuse arguments buffer (save allocations) when calling Dragonfly command from lua scripts.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-10-11 19:19:48 +03:00
Shahar Mike
b1bd2103d7
opt(server): If unspecified, use num_shards == proactor_threads (#1998)
* opt(server): If unspecified, use num_shards == proactor_threads

* Use same config for tests, per Roman's request
2023-10-10 14:12:39 +03:00
Kostas Kyrimis
cbba6f4e11
fix(acl/admin_port): skip acl validation on admin port (#1997)
* skip all acl related functionality on admin port
2023-10-10 08:28:07 +03:00
Kostas Kyrimis
5471827448
update(requirepass): behave as redis requirepass (#1996)
* requirepass also updates ACL default user password
* update config set requirepass to include the new behaviour
* add tests
* fix non existent default user when loading empty files
2023-10-09 16:34:42 +03:00
Vladislav
60ec6a3a27
fix: fix eval in multi interpreter borrow (#1999)
* fix: fix eval in multi interpreter borrow

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-10-07 13:07:06 +03:00
Vladislav
e84d9a65d8
fix(server): Add additional metrics (#1975)
* fix(server): Clean up metrics collection
* feat(server): Replication memory metrics
* fix(server): Limit dispatch queue size

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-10-06 14:16:22 +03:00
Kostas Kyrimis
8b7a43d214
fix(Listener): skip auth when requirepass is empty on http (#1985)
* Fix a small bug on http when username was required even if requirepass was empty
2023-10-03 19:56:19 +03:00
Vladislav
1ea0c39d6a
fix(server): Fix async lua (#1982)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-10-03 16:13:38 +03:00
Kostas Kyrimis
945d3a39e4
chore(Http): change authed username to default and skip auth on metrics (#1950)
* Update Http auth with username default instead of user
* skip auth for /metrics page
* add/improve tests
* fix a bug with admin port requiring auth on http even if nopass was set
* update helio ref
* update listener class to contain its respective Role
* fix http init to only include admin and main listener
2023-10-03 10:45:37 +03:00
Roman Gershman
43d55fa6d7
chore: rename owner to conn (#1973)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-10-01 10:56:18 +03:00
Andy Dunstall
e6b8cd1d76
feat(cmd): add restricted commands flag (#1967) 2023-09-29 16:16:06 +01:00
Roy Jacobson
4ecab6e786
Add a feature flag for partial_sync (#1957) 2023-09-28 13:15:05 +03:00
Shahar Mike
2091f53777
opt(lua): Avoid separate hops for lock & unlock on single shard execution (#1900) 2023-09-22 14:09:17 +03:00