Commit graph

992 commits

Author SHA1 Message Date
ashotland
6fcd15d1e1
Sentinel test debug (#1125)
Sentinal test debug
- Some more logging
- Use PortPicker

Signed-off-by: ashotland <ari@dragonflydb.io>
2023-04-24 10:53:51 +03:00
Roman Gershman
d78a9daaeb
chore: simplify debian package filename. (#1128)
Currently deployed packages have version in the filename which makes it much harder to fetch
using scripts.

This change fixes the filename and also removes some redundant code.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-04-24 10:05:01 +03:00
Roman Gershman
51c5bb7273
feat: Add skeleton code for parser/lexer generation (#1126)
Add initial test for the scanner.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-04-24 09:58:20 +03:00
adiholden
7f56a435c4
bug(server): replicate scripts in stable state (#1114)
* bug(server): replicate scripts in stable state

---------

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-04-23 23:46:51 +03:00
Vladislav
2d501111ea
Fix async lua bugs (#1123)
Fix async lua bugs:
- Not calling toupper before looking up command id
- Wrong error reply context
2023-04-23 14:50:25 +03:00
adiholden
c6d34678f3
fix(test): add wait for all replicas to finish (#1124)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-04-23 14:20:01 +03:00
Vladislav
520e66c4b7
fix: replace journal executor reply builder (#1120)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-04-23 13:27:47 +03:00
Roman Gershman
78a471582f
chore: adding polishes for reading cgroup files (#1121)
Specifically, do not check-fail when reading cgroup file path.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-04-23 11:06:46 +03:00
Vladislav
ba19cc1047
docs: describe command squashing (#1119)
Signed-off-by: Vladislav <vlad@dragonflydb.io>
2023-04-22 09:02:36 +03:00
Vladislav
74a1454409
Collect errors from async lua calls (#1092)
Error collection from async calls
2023-04-22 09:02:22 +03:00
Vladislav
71147c20a9
Dispatch queue memory optimizations (#1103)
Dispatch queue entry optimizations
2023-04-22 09:02:07 +03:00
talbii
2d73b2bfb0
fix: container limits (#1109)
* added checks to limit memory to container\'s limit
* some fixes, move all logic to a single function
2023-04-21 00:52:16 +03:00
Roy Jacobson
46a2435865
fix(tests): Disable 'dbfilename' unless explicitly given in the test (#1117) 2023-04-20 13:11:49 +03:00
adiholden
3b3350081c
chore: enable self hosted runner (#1116)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-04-20 11:36:04 +02:00
adiholden
9a515cbe67
docs(readme) : doc flagfile (#1110)
Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-04-20 11:28:22 +03:00
iko1
c2d32f9d68
fix(server): json.set should add missing keys and add missing cmd... (#1065)
fix(server): json.set should add missing keys and add missing command flags

Signed-off-by: iko1 <me@remotecpp.dev>
2023-04-20 11:20:00 +03:00
Roy Jacobson
6632261a2d
fix(server): Prevent preemption inside SerializeBucket (#1111)
* fix(server): Prevent preemption inside SerializeBucket

* Modifications after speaking to Adi
2023-04-20 10:27:47 +03:00
Roy Jacobson
246f6093db
feat(server): Save on shutdown (#1086)
* feat(server): Save snapshot on shutdown

* CR

* Change save on shutdown to be conditional on --dbfilename.

* Support SHUTDOWN [NO]SAVE and fix unit test

* Better wait for DB loading

* Fix DF format loading state bug

* Fix some fallout from auto save
2023-04-20 06:30:42 +02:00
Roy Jacobson
dd97b36965
fix(server): Better error reporting when opening snapshot files (#1112) 2023-04-19 15:34:59 +03:00
Roy Jacobson
1319918476
fix(server): Don't allow directory escapes in dbfilename (#1105)
* fix(server): Don't allow directory escapes in dbfilename

* Just disallow directories completely
2023-04-18 21:21:23 -07:00
Roy Jacobson
002ec8a621
fix(parser): Don't accept NILs as command arguments in server mode (#1101)
* fix(parser): Don't accept NILs as command arguments in server mode
2023-04-18 17:44:36 +03:00
Tarun Pothulapati
2033b671ef
fix(helm): Allow prometheus rules fields to be empty (#1100)
This commit fixes a rendering error where when the rules are
empty, the helm chart fails to render. By using the correct
indentation, the chart will render correctly.
2023-04-18 20:08:51 +05:30
Vladislav
77e18f0463
fix: report errors from commands with redis.call (#1108)
Redis call now directly reports erros
2023-04-18 17:29:07 +03:00
Roy Jacobson
1382ed1c37
chore: Allow passing empty strings as flag arguments in DflyInstance (#1104)
For example, this is needed to disable auto save on shutdown with --dbfilename=""
2023-04-18 12:48:30 +03:00
Roy Jacobson
d9f45f369a
fix(server): Allow interrupting REPLICAOF commands during full sync (#1058)
* fix(server): Allow interrupting REPLICAOF commands during full sync
* fix(server): Fix a race condition and UB in a debug assert
2023-04-18 11:44:44 +03:00
Vladislav
0e7d137046
fix: fix default instance port blocking in pytest (#1102)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-04-18 00:12:38 +03:00
Vladislav
c65073eca9
Refactor connection (#1098)
Refactors dragonfly connection class
2023-04-17 21:02:48 +03:00
Roy Jacobson
c952251381
feat(server): Insert timestamp into snapshot names explicitly (#1028)
* feat(server): Insert timestamp into snapshot names explicitly

* Whenever the snapshot filename contains '{timestamp}', it will be substituted with the current local time.
* Default snapshot name is now "dump-{timestamp}"
* InferLoadFile: Modify to recognize "{timestamp}" files correctly.
* ServerFamily::Load: Change extension 'CHECK' into a non-terminating error because it's user-visible
* ServerFamily::DoSave: Add sanity check for the filename extension.

Signed-off-by: Roy Jacobson <roy@dragonflydb.io>

* resolve CR comments

* Add comment about glob sorted output

* Fix InferLoadFile and fix its tests

* Simplify filename behavior with the .dfs format

---------

Signed-off-by: Roy Jacobson <roy@dragonflydb.io>
2023-04-17 14:59:44 +03:00
Roy Jacobson
cb17f92ad8
feat(server): Support IPv6 addresses for replication (#1087)
* feat(server): Support IPv6 addresses for replication

* break earlier on IPv4
2023-04-17 07:55:11 +03:00
Roman Gershman
e17fb41dee
fix: systemd configuration (#1097)
1. Tune some security directives.
2. Fix the flags file that mistakenly configured dragonfly to store its dump files into /run (tmpfs).

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-04-16 06:41:02 -07:00
Roman Gershman
74e94ef9bb
feat: add oom insertion rejections (#1096) 2023-04-16 10:23:09 +03:00
Vladislav
b345604226
fix: Remove incremental locking (#1094) 2023-04-15 06:59:19 -07:00
Chaka
023abfa46e
feat(server): Support CONFIG GET [databases|maxmemory] (#1093)
Fix #1046

Signed-off-by: chakaz <chakaz@chakaz>
Co-authored-by: chakaz <chakaz@chakaz>
2023-04-15 11:41:20 +03:00
Roman Gershman
fcb1c7bd55
fix: bugs in rdb code (#1079)
1. Pull newest helio that fixes 32-bit overflow in IoBuf.
2. Allow responsiveness when loading huge sets and maps.
3. Break array of blobs into segments of limited size so that
   we won't allocate billion byte arrays when handling large (h)sets.
   It reduces pressure on allocator when loading millions of items.

Fixes #1076.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Co-authored-by: Roy Jacobson <roy@dragonflydb.io>
2023-04-13 13:26:58 +03:00
Roy Jacobson
fc66dbb2cf
fix(tests): Don't return the same port from PortPicker.get_available_port twice (#1083) 2023-04-13 09:56:46 +03:00
Vladislav
70cf436c05
Lua script async calls (#1070)
Introduces squashing for scripts and a new `redis.acall` command for async commands
2023-04-12 23:37:25 +03:00
Vladislav
282c168d34
fix: Update cntx->cid on multi-tx'es (#1081)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-04-12 23:28:31 +03:00
Chaka
f013699613
Revert to use CachePrevValueIfNeeded(), but let it call GetString() if needed. (#1080)
I believe this is a better API as it's harder to mistakenly use.

Signed-off-by: chakaz <chakaz@chakaz>
Co-authored-by: ashotland <ari@dragonflydb.io>
2023-04-12 22:19:06 +03:00
ashotland
5fb5f54885
fix(regression-tests): Add PortPicker (#1078)
* fix(regression-tests): Add PortManager

Add PortManager to find and use available ports in regression tests.
Use it in redis_replicaiton_test.
---------

Signed-off-by: ashotland <ari@dragonflydb.io>
2023-04-12 19:14:29 +03:00
Roman Gershman
ee24029148
feat: enable multi-tx squashing. (#1074)
Make blpop test pass with squashing. The problem was that the txid was not properly reported
back to the testing code.

Also, fixed redundant codepath for defragmentation.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-04-12 16:15:35 +03:00
ashotland
59f1ac6701
fix(regression-test): Delete tests/dump.rdb (#1077)
fix(regression-test): Delete dump.rdb

This fails redis to start in redis_replicaiton_test,
I assume it was added accidently.

Signed-off-by: ashotland <ari@dragonflydb.io>
2023-04-12 14:13:39 +03:00
Chaka
689b4e1a67
Only copy previous value's string when needed. (#1075)
Previously we called GetString() every time SET was used, which is
unneeded in the vast majority of cases.

Signed-off-by: chakaz <chakaz@chakaz>
Co-authored-by: chakaz <chakaz@chakaz>
2023-04-12 09:42:33 +03:00
Vladislav
c774aee6fb
fix: fix lua monitor pytest (#1073)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-04-11 23:01:34 +03:00
Vladislav
90a606851a
chore: Split DispatchCommand into validation and main part (#1071)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-04-11 15:41:49 +03:00
ashotland
d9c9d9fc72
fix(server): Fix resp3 for pubsub commands (#1068)
Add a PUSH ('>') Collection type and use it for pubsub commands.

Signed-off-by: ashotland <ari@dragonflydb.io>
2023-04-11 14:04:50 +03:00
Roy Jacobson
8621442a5f
chore: update helio (#1069) 2023-04-11 13:31:11 +03:00
Vladislav
015ed622c5
fix(server): Optimize StoredCmd (#1053)
Opmitize StoredCmd to allow inline storage
2023-04-11 10:14:36 +03:00
Roman Gershman
0ff22c6078
chore: disable self-hosted runner (#1067) 2023-04-10 22:24:34 +03:00
Vladislav
a12ddfe108
Remove cmd name from args (#1057)
chore: remove cmd name from the list of arguments

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Roman Gershman <roman@dragonflydb.io>
2023-04-10 14:14:52 +03:00
adiholden
8600eacdc4
bug(sse2neon): fix sanitizer crash in _mm_loadu_si128 (#1037)
* bug(sse2neon): fix sanitizer crash in _mm_loadu_si128

Signed-off-by: adi_holden <adi@dragonflydb.io>

* remove static

Signed-off-by: adi_holden <adi@dragonflydb.io>

---------

Signed-off-by: adi_holden <adi@dragonflydb.io>
2023-04-10 10:56:58 +03:00