dragonflydb / dragonflydb/dragonfly

Single-shot arena data heap destruction

Open
#8,302 0 comments 0 reactions 1 assignee Claimed by @kostasrim View on GitHub
enhancement
Dominant language
C++
Stars
31.5k
Forks
1.3k
Avg merge
1d 10h
Merged PRs (30d)
137

Description

# Slow shutdown on large data heaps with million of objects

We need to call the destructor for each object. For k objects, we call O(k) destructors. The solution is to `reap` the heap in one shot via `mi_heap_destroy`. This `skips` the destructors and returns the backing pages in one-shot.

This is also applicable to `flushall` which will also be implemented as part a separate round of PR's.

For this to work it requires:

* No other use of the data heap after the heap is reaped
* Call explicitly any destructor that needs to a) notify other fibers b) clean up resources like files, etc d) that needs to deallocate an allocation by the backing heap (global new)
* Clean up the thread locals

I prototyped this (along with other things) here: https://github.com/dragonflydb/dragonfly/pull/8239

Shutdown from 75s(timedout and sig-killed by the python harness) took `0.5s` 😄

# Tasks - Group 1 - data and backing heap cleanups/fixes

- [ ] protocol client resp parser should use the backing heap (currently uses the data heap)
- [ ] topk should use only the data heap and not the backing
- [ ] qlist tiering params should use the data heap
- [x] sorted map should use only the data heap
- [ ] dbtable and dbslice (small objects) should be allocated on the data heap. The reason for this is that otherwise we need to call their destructors (since they are allocated via the backing heap -- see (c) in previous section). We can't reap the heap before we call those destructors but calling them (~dbslice) would mean we do O(k) deletions (would call mi_free on the dash table!). `To solve this cycle`, dbtable and dbslice become data heap objects and released one-shot via `mi_destroy_heap`.

# Tasks - Group 2 - actual shutdown path

- [x] patch mi_malloc_destroy
- [ ] EngineShard is a composite objects whose some parts are allocated via the backing heap. Add `PrepareForSingleShotShutdown` that covers things that must be deallocated explicitly (backing heap destructors, file resources etc) and then call `mi_heap_destroy`. Also reset the thread locals afterwards

# Future

- [ ] prototype one shot flushall
- [ ] clean up engine shard. Everything in it should go through the data heap (although I must verify that we don't need the backing for all the allocations). That way we will remove the `PrepapareForSingleShotShutdown` and just call `mi_heap_destroy` in one go.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.