dragonflydb / dragonflydb/dragonfly
Single-shot arena data heap destruction
- Dominant language
- C++
- Stars
- 31.5k
- Forks
- 1.3k
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 137
Description
# Slow shutdown on large data heaps with million of objects
We need to call the destructor for each object. For k objects, we call O(k) destructors. The solution is to `reap` the heap in one shot via `mi_heap_destroy`. This `skips` the destructors and returns the backing pages in one-shot.
This is also applicable to `flushall` which will also be implemented as part a separate round of PR's.
For this to work it requires:
* No other use of the data heap after the heap is reaped
* Call explicitly any destructor that needs to a) notify other fibers b) clean up resources like files, etc d) that needs to deallocate an allocation by the backing heap (global new)
* Clean up the thread locals
I prototyped this (along with other things) here: https://github.com/dragonflydb/dragonfly/pull/8239
Shutdown from 75s(timedout and sig-killed by the python harness) took `0.5s` 😄
# Tasks - Group 1 - data and backing heap cleanups/fixes
- [ ] protocol client resp parser should use the backing heap (currently uses the data heap)
- [ ] topk should use only the data heap and not the backing
- [ ] qlist tiering params should use the data heap
- [x] sorted map should use only the data heap
- [ ] dbtable and dbslice (small objects) should be allocated on the data heap. The reason for this is that otherwise we need to call their destructors (since they are allocated via the backing heap -- see (c) in previous section). We can't reap the heap before we call those destructors but calling them (~dbslice) would mean we do O(k) deletions (would call mi_free on the dash table!). `To solve this cycle`, dbtable and dbslice become data heap objects and released one-shot via `mi_destroy_heap`.
# Tasks - Group 2 - actual shutdown path
- [x] patch mi_malloc_destroy
- [ ] EngineShard is a composite objects whose some parts are allocated via the backing heap. Add `PrepareForSingleShotShutdown` that covers things that must be deallocated explicitly (backing heap destructors, file resources etc) and then call `mi_heap_destroy`. Also reset the thread locals afterwards
# Future
- [ ] prototype one shot flushall
- [ ] clean up engine shard. Everything in it should go through the data heap (although I must verify that we don't need the backing for all the allocations). That way we will remove the `PrepapareForSingleShotShutdown` and just call `mi_heap_destroy` in one go.
Contributor guide
Assessment
This issue has not been assessed yet.