* Revert "Revert "chore: get rid of kv_args and replace it with slices to full_… (#3024)"
This reverts commit 25e6930ac3.
Fixing the performance bug caused by applying equality operator
on two spans inside ShardArgs::Iterator::operator==
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Vladislav Oleshko <vlad@dragonflydb.io>
The main change is in tx_base.* where we introduce ShardArgs slice that
is only forward iterable. It allows us to go over sub-ranges of the full arguments
slice or read an index of any of its elements.
Since ShardArgs provide now indices into the original argument list we do not need to build the reverse index in transactions.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Done in preparation to make ShardArgs a smart iterable type,
but currently it's just a wrapper aroung ArgSlice.
Also refactored common.{h,cc} into tx_base.{h,cc}
In addition, fixed a bug in key tracking, where we wrongly created weak_ref
in a shard thread instead of doing this in the coordinator thread.
Finally, identified another bug (not fixed yet) where we track all the arguments
instead of tracking keys only.
Besides this, no functional changes around the moved code.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
A self-laundering iterator will enable us to, eventually, yield from fibers while holding an iterator. For example:
```cpp
auto it1 = db_slice.Find(...);
Yield(); // Until now - this could have invalidated `it1`
auto it2 = db_slice.Find(...);
```
Why is this a good idea? Because it will enable yielding inside PreUpdate() which will allow breaking down of writing huge entries in small quantities to disk/network, eliminating the need to allocate huge chunks of memory just for serialization.
Also, it'll probably unlock future developments as well, as yielding can be useful in other contexts.
* chore: implement path mutation for JsonFlat
This is done by converting the flat blob into mutable c++ json
and then building a new flat blob.
Also, add JsonFlat encoding to CompactObject class.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: Del and NUMINCRBY use json::Path
Also, fix various protocol bugs when we sent simple string
instead of sending bulk strings.
Fixed a typo in path.cc that lead to a data race bug.
Finally, flip the flag in regression tests to start covering json::Path code
and added test coverage for the data race bug
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: switch to self-built flatbuffers
This should solve the linker errors on MacOs build.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
feat: a test with flat buffers
Also, add an experimental flag `--experimental_flat_json` that allows writing json objects as flat strings using
flexibuffers.
The experiment shows that `debug populate 100000 a 10 type json elements 30`
uses almost 3 times less memory than with native jsoncons objects.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Fix linker dependencies - now dfly_core depends on jsonpath.
jsonpath does not depend on CompactObject anymore.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* feat: wire json::Path into the server
The feature is disabled under a flag and it covers read operations for now.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: switch json object to pmr allocator
1. Move json object creation code to the shard-thread inside rdb_load
2. Now json_family never references "json" type, always dfly::JsonType
3. Switch JsonType to pmr::json.
4. Make sure we pass the correct memory_resource when creating json object from string.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* WIP: Auto `PostUpdate()`
* More `Find()` uses
* Final touches
* Fixes
* Fix bug and allow reassigning
* Rename to AutoUpdater
* Fix and add DCHECK
* Also check deletion count
* Use ccache instead of sccache
* Try to upgrade Helio
* off64_t
* off64_t
* Revert changes to CI
Specifically, allocate only a single blob when returning multiple entries from a shard.
In addition, refactor and unify MGetResponse between string family code and ReplyBuilder code.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
1. Add acl categories to each command
2. Extend `facade::CommandId` to include acl category
3. Add dragonfly extensions to acl categories for modules like search, json etc (since modules in redis do not have acl categories by default but in dragonfly these are not implemented as modules)
update denyoom flag for all commands
DENYOOM command flag marks which commands will be denied when memory usage crosses the "red zone".
Signed-off-by: adi_holden <adi@dragonflydb.io>
chore: remove cmd name from the list of arguments
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Co-authored-by: Roman Gershman <roman@dragonflydb.io>
