mirrors/dragonfly
1
0
Fork 0
mirror of https://github.com/dragonflydb/dragonfly.git synced 2025-05-11 18:35:46 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 5
1762 commits 76 branches 100 tags 60 MiB
Commit graph

1762 commits

This branch
This branch
All branches
Author SHA1 Message Date
Andy Dunstall
8f28a3826b
feat(aws): add web identity provider (#2041) 
chore: update helio
 2023-10-26 09:33:08 +00:00
Shahar Mike
424c65b4e5
fix(bptree): Reverse iteration (#2075) 
Co-authored-by: Roman Gershman <roman@dragonflydb.io>
 2023-10-25 20:07:57 +00:00
Shahar Mike
b2e989a4fb
fix(server): Fix crash when connection uses multiple lua scripts (#2073) 2023-10-25 19:44:09 +00:00
Kostas Kyrimis
67b8908805
chore(server-family): do not log SETINFO (#2065) 
* chore(server-family): do not log SETINFO

* address gh comments
 2023-10-25 18:15:00 +03:00
adiholden
254fb2dd5a
feat(server): add memory stats for snapshot save (#2072) 
* feat(server): add memory stats for snapshot save

Signed-off-by: adi_holden <adi@dragonflydb.io>
 2023-10-25 17:36:10 +03:00
Vladislav
5eed2bfe11
fix(search): Persist field flags (#2069) 
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
 2023-10-25 13:22:40 +03:00
Uku Loskit
e9427dbbbc
feat: Implement options for EXPIRE command (#2051) 
Add NX, XX, LT, GT options to EXPIRE, EXPIREAT, PEXPIRE and PEXPIREAT

Signed-off-by: Uku Loskit <ukuloskit@gmail.com>
 2023-10-25 11:26:33 +03:00
Roman Gershman
eefd0c7808
chore: add more debug info if a deadlock is caught in unit tests (#2062) 
Addresses #2056

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
 2023-10-24 18:15:00 +03:00
Andy Dunstall
1a813ce1b7
feat(aws): add https support (#2061) 2023-10-24 11:31:05 +01:00
Roman Gershman
dc025e5f94
fix: inject our own parser for slowlog get (#2059) 
Also adjust debug logs to be less verbose.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
 2023-10-24 11:02:51 +03:00
Andy Dunstall
124bafc06b
feat(tls): support runtime tls reconfig (#2047) 
* feat(tls): add tls reconfig

* feat(config): error if multiple config params given

* tls: move ctx ref to connection
 2023-10-23 16:35:39 +00:00
Roman Gershman
3095d8a168
feat: add 'memory decommit' command (#2058) 
This command forces the memory manager to decommit memory pages back to the OS.
In addition, fixed some positional bugs in "memory malloc-stats"

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
 2023-10-23 16:49:00 +03:00
Kostas Kyrimis
dd2cb1def1
fix(regTests): properly handle other error codes on timeouts (#2057) 2023-10-23 16:35:29 +03:00
Roy Jacobson
313501d987
fix(sock): Use the updated cancellation cb interface. (#1940) 
* fix(sock): Use the updated cancellation cb interface.

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Vladislav Oleshko <vlad@dragonflydb.io>
 2023-10-23 13:08:06 +03:00
Vladislav
67bb3970be
chore(search): Add exclusive range option (#2055) 
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
 2023-10-23 13:07:40 +03:00
Roman Gershman
15f54be762
chore: measure rss memory and track its peak (#2054) 
Up until know we did not have cached rss metric in the process.
This PR consolidates caching of all values together inside the EngineShard periodic fiber
code. Also, we know expose rss_mem_current that can be used internally for identifying
memory pressuring periods during the process run.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
 2023-10-22 19:01:40 +00:00
Roman Gershman
d9cb7453fb
wrap dragonfly_test with gdb inside a CI (#2050) 
chore: run dragonfly_test with epoll under gdb

Also, update helio that provide a stacktrace under musl libc (alpine linux).
This version of helio updates absl version as well.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
 2023-10-22 20:05:27 +03:00
Vladislav
2cb7d30603
fix: skip setting tcp_nodelay for unix domain sockets (#2033) 
* fix handling of unix domain sockets

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
 2023-10-22 11:00:51 +03:00
Roman Gershman
83d5b849a3
fix: localize timeout in test_cancel_replication_immediately test (#2048) 2023-10-22 08:44:00 +03:00
Vladislav
9e8d886ce9
chore(search): add redis-om test (#2049) 
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
 2023-10-21 14:22:04 +03:00
Roman Gershman
215c037e41
feat: implement fieldttl for the hash family (#2040) 
Fix some corner case bug with set family as well.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
 2023-10-20 21:51:24 +03:00
Vladislav
4a2dd30886
fix: Don't crash on reaching hnsw capacity (#2046) 
* fix: Don't crash on reaching hnsw capacity

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
 2023-10-20 19:36:52 +03:00
Kostas Kyrimis
1d02e12ad1
chore: call debug stacktrace on SIGUSR1 (#2012) 
* add macro to install a signal handler that prints the contents of debug stacktrace on SIGUSR1
* add this on regTests
 2023-10-20 10:50:55 +03:00
Kostas Kyrimis
64841efeed
chore(regTests): print logs when regTests timeout (#2031) 
* add a python script to print the most recent log
* if CI timeouts, print the most recent log
* replace global timeout with timeout command
* upload all logs on failure()
* print uid + port + the log files for each df instance
 2023-10-20 10:50:19 +03:00
Shahar Mike
bc0064e94d
opt(server): Migrate connections which invoke Lua scripts to their target thread. (#2037) 
This is enabled by default, but can be disabled via `--migrate_connections=false`.

Measuring with BullMQ benchmark I see a gain of almost 10% in
throughput. I haven't measured, but it's supposed to also reduce
latency.
 2023-10-19 22:31:07 +03:00
Uku Loskit
395f65ddd2
feat(acl): Implement ACL GENPASS (#2013) 2023-10-19 15:37:24 +00:00
iko1
954e5a0a80
feat(acl): add config set for aclfile (#1978) 2023-10-19 10:15:38 +00:00
Erick Velez
7f03be9439
feat: implement SINTERCARD (#1994) 
Signed-off-by: Erick Velez <erickvelez7@gmail.com>
 2023-10-18 19:26:08 +03:00
zixuan zhao
655959af37
fix: evict open connections for users not present in the updated ACL (#2019) 2023-10-18 13:45:56 +00:00
Roman Gershman
2392eae8a3
chore: add a debug check in slowlog (#2039) 2023-10-18 11:28:00 +00:00
Vladislav
22a703de38
feat(server): Support COMMAND INFO (#2035) 
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
 2023-10-17 18:33:55 +03:00
Vladislav
4b387cebe2
fix(server): Exclude eval from pipeline squashing (#2027) 
* fix(server): Exclude eval from pipeline squashing

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
 2023-10-16 17:48:42 +03:00
romange
4e0126d72e chore(helm-chart): update to v1.11.0 2023-10-16 10:40:31 +00:00
Tarun Pothulapati
b1ba29f9c7
fix(ubuntu-prod): Set suexec hash correctly (#2029) 2023-10-16 13:31:20 +03:00
Roman Gershman
c6946d561c
feat: add fieldttl command that returns the ttl of the member (#2026) 
Currently implemented only for saddex.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
 2023-10-16 12:12:00 +03:00
Vladislav
6c1e9fcefc
fix(search): Use double numeric values (#2015) 
* fix(search): Use double numeric values

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
 2023-10-16 10:19:30 +03:00
Shahar Mike
9e2a9fd5e5
fix(server): Fix shardlocal eval with round-robin prefix (#2024) 2023-10-15 19:36:41 +03:00
Roman Gershman
1cb8be26ea
chore: remove .DS_Store file (#2025) 2023-10-15 19:27:07 +03:00
Shahar Mike
838d01af32
fix(server): Start periodic fiber only once init completes (#2023) 2023-10-15 14:56:55 +03:00
Roman Gershman
bcfd1863c7
fix: reject zset variadic commands with 0 keys (#2022) 
Fixes the assertion failure as reported by #1994.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
 2023-10-15 14:34:04 +03:00
Vladislav
826b1d81a3
chore: Enable squashing by default (#2011) 
* chore: Enable squashing
 2023-10-15 12:17:52 +03:00
Shahar Mike
96b8dd9ca6
opt(server): Allow round-robin keys between shards based on prefix (#2016) 
This will allow some use cases with few busy keys to distribute load
more evenly between threads.

Idea by @dranikpg.

To calculate how many entries are needed in the table I used the
following quick-n-dirty code, to reach <2.5% collision with 100 keys:

```cpp
bool Distribute(int balls = 100, int bins = 100) {
  vector<int> v(bins);
  for (int i = 0; i < balls; ++i) {
    v[rand() % v.size()]++;
  }

  for (int v : v) {
    if (v >= 2) {
      return true;
    }
  }

  return false;
}

int main(int argc, char** argv) {
  int has_2_balls = 0;
  constexpr int kRounds = 1'000'000;
  for (int i = 0; i < kRounds; ++i) {
    has_2_balls += Distribute(100, 100'000);
  }
  cout << has_2_balls << " rounds had 2+ balls in a single bin out of " << kRounds << endl;
}
```
 2023-10-15 11:52:44 +03:00
Roman Gershman
acc00b77b3
feat: allow extracting the expiry time from string_set (#2020) 
We introduce `iterator Find(member)` function
as well as iterator members to actually get the expiry time.
 2023-10-15 09:56:17 +03:00
Vladislav
cb9a45f2a9
fix(server): Don't recompute shard for squashed stub tx (#2017) 
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
 2023-10-15 09:27:58 +03:00
Uku Loskit
6a75c6ddc5
feat: Implement slowlog (#1956) 
Implement slowlog

Signed-off-by: Uku Loskit <ukuloskit@gmail.com>
 2023-10-13 23:39:35 +03:00
Kostas Kyrimis
32a0baa62c
chore(regTests): separate build and reg tests timeouts (#2006) 
* remove timeout from build step
* add 45 min timeout for execution of regTests
 2023-10-13 09:01:12 +03:00
Jongwoo Han
4ad2c49523
chore: Replace deprecated command with environment file (#2010) 
Signed-off-by: Jongwoo Han <jongwooo.han@gmail.com>
 2023-10-12 17:22:31 +03:00
Kostas Kyrimis
76a030b1a7
chore(regTests): add basic module logging and print when tests fail (#2005) 
* add vmodule level 1 logging for df instances
* print logs when df instance fails to stop
 2023-10-12 14:38:35 +03:00
Roman Gershman
ec87114f66
fix: buffer overrun when passing long command name from lua (#2008) 
Also, few additional changes that do not affect functionality.
1. make sure passed arguments to DispatchCommand are `\0` delimited
   during pipelining.
2. extend lua malloc hook to call precise functions - to help with cpu profiling.
3. reuse arguments buffer (save allocations) when calling Dragonfly command from lua scripts.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
 2023-10-11 19:19:48 +03:00
Roman Gershman
c6f8f3882a
chore: add balls and bins simulator (#2001) 
* chore: add balls and bins simulator

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

* Update balls_bins.py

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
 2023-10-11 01:18:29 +03:00