fix: do not check-fail OpRestore
In some rare cases we reach inconsistent state inside OpRestore where a key already exists, though it should not.
In that case log the error instead of crashing the server. In addition, we update the existing entry to the latest restored value.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: add ability to track connections stuck at send
Add send_delay_seconds/send_delay_ms metrics.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Signed-off-by: Roman Gershman <romange@gmail.com>
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
Before: if socket data arrived in small bits, then CheckForHttpProto would grow
io_buf_ capacity exponentially with each iteration. For example, test_match_http test
easily causes OOM.
This PR ensures that there is always a buffer available - but it grows linearly with the input size.
Currently, the total input in CheckForHttpProto is limited to 1024.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: factor out rdb_load utilities into separate files
rdb_load.cc is huge and contains many auxillary classes.
This PR moves DecompressImpl and ErrorRdb code into detail/
It also fixes minor bugs around error conditions with de-compression:
a. Do not check-fail on invalid opcode and return error_code instead.
b. Print correctly LZ4 errors.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: fixes
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: improve parser state machine
1. Separate argument type parsing from argument parsing itself.
2. Handle strings of length 1.
This is done in preparation of improving the parser contract -
so that when it returns INPUT_PENDING, it consumes the entire input.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: add active time to stream consumers
Adjust XINFO command to output active-time property.
Store active-time and switch to RDB_TYPE_STREAM_LISTPACKS_3 if FLAGS_stream_rdb_encode_v2
is enabled.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Now rdb_load supports RDB_TYPE_STREAM_LISTPACKS, RDB_TYPE_STREAM_LISTPACKS_2 and RDB_TYPE_STREAM_LISTPACKS_3 formats.
rdb_save still saves with RDB_TYPE_STREAM_LISTPACKS format - we want to release the DF version that can load everything first, and
then update the replicaion format in the next versions.
Also, update rdb_test.cc
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: let resp parser provide more useful logs
1. More warning logs around bad BAD_ARRAYLEN messages
2. Lift the restriction around big bulk strings and log a warning instead.
3. Pull helio
Probably fixes#4213
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: Fix `test_rss_used_mem_gap` for all types
The test fails when it checks the gap between `used_memory` and
`object_used_memory`, by assuming that all used memory is consumed by
the `type` it `DEBUG POPULATE`s with.
This assumption is wrong, because there are other overheads, for example
the dash table and string keys.
The test failed for types `STRING` and `LIST` because they used a larger
number of keys as part of the test parameters, which added a larger
overhead.
I fixed the parameters such that all types use the same number of keys,
and also the same number of elements, modifying only the element sizes
(except for `STRING` which doesn't have sub-elements) so that the
overall `min_rss` requirement of 3.5gb still passes.
Fixes#3723
* threshold
* list
* comments test assert
* previous numbers
* ???
We have cascading failures in cluster tests because on assertion failures the nodes are not properly cleaned up and subsequent test cases that use the same ports fail. I added a monotonically increasing port generator to mitigate this effect.
* remove DbSlice mutex
* add ConditionFlag in SliceSnapshot
* disable compression when big value serialization is on
* add metrics
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
The bug:
calling lua_error does not return, instead it unwinds the Lua call stack until an error handler is found or the
script exits. This lead to memory leak on object that should release memory in destructor.
Specific example is the absl::FixedArray<string_view, 4> args(argc); which allocates on heap if argc > 4. The free was not called leading to memory leak.
The fix:
Add scoping to to the function so that the destructor is called before calling raise error
Signed-off-by: adi_holden <adi@dragonflydb.io>
This PR syncs some of the improvements that were introduced in streams in Redis 7.2.3 OSS.
1. verify xsetid against max deleted id in the stream
2. Implement precise memory measurement of streams for "memory usage" command.
3. Always compact nodes in stream listpacks after creating new nodes.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
