The problem: apparently, jsoncons uses strtod by default when parsing doubles.
On some platforms (alpine/musl) this function uses lots of stack, which potentially can lead to stack corruption.
This PR configures jsoncons to use std::from_chars that is more efficient and less stack hungry.
The single include point to consume jsoncons/json.hpp should be "core/json_object.h"
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: change Namespaces to be a global pointer
Before the namespaces object was defined globally.
However it has non-trivial d'tor that is being called after main exits.
It's quite dangerous to have global non-POD objects being defined globally.
For example, if we used LOG(INFO) inside the Clear function , that would crash dragonfly on exit.
Ths PR changes it to be a global pointer.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* feat: introduce metrics/logs of when pipelining is being throttled
Fixes#3999 following up on discussion at #3997.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Now unit tests will run the same Hearbeat fiber like in prod.
The whole feature was redundant, with just few explicit settings of maxmemory_limit
I succeeeded to make all unit tests pass.
In addition, this change allows passing a global handler that is called by heartbeat from a single thread.
This is not used yet - preparation for the next PR to break hung up replication connections on a master.
Finally, this change has some non-functional clean-ups and warning fixes to improve code quality.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* feat(namespaces): Initial support for multi-tenant #3050
This PR introduces a way to create multiple, separate and isolated
namespaces in Dragonfly. Each user can be associated with a single
namespace, and will not be able to interact with other namespaces.
This is still experimental, and lacks some important features, such as:
* Replication and RDB saving completely ignores non-default namespaces
* Defrag and statistics either use the default namespace or all
namespaces without separation
To associate a user with a namespace, use the `ACL` command with the
`TENANT:<namespace>` flag:
```
ACL SETUSER user TENANT:namespace1 ON >user_pass +@all ~*
```
For more examples and up to date info check
`tests/dragonfly/acl_family_test.py` - specifically the
`test_namespaces` function.
* remove helper.h and helper.cc and move them to AclFamily class
* remove global tables from acl_commands_def and move them to AclFamily
Signed-off-by: kostas <kostas@dragonflydb.io>
* print categories and commands in lower case instead of capital case
* fix a bug of default user inheriting the wrong acl rules on new connections
* move keys position to be after password when printed from an acl command
* chore: Introduce pipeline back-pressure
Also, improve synchronization primitives and replace them with
thread-local variations.
Before the change, on my local machine with the dragonfly running with 8 threads,
`memtier_benchmark -c 10 --threads 8 --command="PING" --key-maximum 100000000 --hide-histogram --distinct-client-seed --pipeline=20 --test-time=10`
reached 10M qps with 0.327ms p99.9.
After the change, the same command showed 13.8M qps with 0.2ms p99.9
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: get rid of lock keys
1. Introduce LockTag a type representing the part of the key that is used for locking.
2. Hash keys once in each transaction.
3. Expose swap_memory_bytes metric.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
The main change here is introduction of the strong type LockTag
that differentiates from a string_view key.
Also, some testing improvements to improve the footprint of the next PR.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* feat(server): Allow configuration of hashtag extraction
Before this PR: Hashtags, if enabled, meant the text between `{` and `}`
in the key (if exists and non-empty).
After this PR:
* Hashtags can _still_ be enabled / disabled
* Hashtag open / close char can be specified (and can be the same), like `:` popular with BullMQ
* Hashtag can include `N` closing tags to skip, like `{a}b}c}d` with `2` will return `a}b}c`.
This will allow some existing systems to migrate without code changes in
client code.
* chore: LockTable tracks fingerprints of keys
It's a first step that will probably simplify dependencies in many places
where we need to keep key strings for that. A second step will be to reduce the CPU load
of multi-key operations like MSET and precompute Fingerprints once.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore(transaction): Simplify armed state
Remove atomic is_armed variable and turn it into a regular local state flag. This is now possible because we have clearly defined phases with the phased barrier and baton barrier for blocking commands
---------
Signed-off-by: Vladislav <vlad@dragonflydb.io>
chore: UpdateHelio dependency
Add support for asan/ubsan checkers in our dev environment.
Remove more clang warnings.
Once we fix all the problems we will enable them in our CI as well.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* fix: Remove a stale reference to blocking watch queue
1. Remove the duplicated FinalizeWatched function
2. Identify the case where we delete the watched queue while we may still have awakedened_keys pointing to it.
3. Add a test reproducing the issue of having in awakened_keys an untangled key.
Properly fixes#2514
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
This should reduce allocations in a common case (not multi).
In addition, rename Transaction::args_ to kv_args_.
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Vladislav <vlad@dragonflydb.io>
1. How many transactions we processed by type
2. How many transactions we processed by width (number of unique shards).
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
fixes https://github.com/dragonflydb/dragonfly/issues/2139
This is part two that implements the logic which notifies tracking clients by sending invalidation messages:
- The client tracking state is set by CLIENT TRACKING subcommand as well
as upon client disconnection.
- Track the keys of a readonly command by maintaining mapping that maps
keys to the sets of tracking clients.
- Send invalidation messages to clients when their tracked keys are
updated.
- Make PerformDeletion a member function of DbSlice, and send
invalidation message within the function.
- Mock the function for sending invalidation message to avoid test
crash due to lack of real listener in the testing framework.
- Add functional (some) tests for client tracking based on the mocked interfaces.
---------
Signed-off-by: Yue Li <61070669+theyueli@users.noreply.github.com>
* Update Http auth with username default instead of user
* skip auth for /metrics page
* add/improve tests
* fix a bug with admin port requiring auth on http even if nopass was set
* update helio ref
* update listener class to contain its respective Role
* fix http init to only include admin and main listener
