Add high water mark to MemoryManager
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and *why* for this feature, in addition to the *what*)
Following conversations from slack it would be useful to have a public high water mark attribute for the memory used by a query. Then this would be able to be used by system that use datafusion, such as IOx or datafusion-tui.
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
I havent had the chance to look into this yet so im not sure the practicality of it but maybe an attribute like `last_query_peak_memory` being added to `MemoryManager`.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
Contributor guide
Research direction
Start by locating MemoryManager and reviewing how it tracks memory for a query. Define how a public high-water-mark attribute such as last_query_peak_memory should behave, then verify that it reports the query's peak memory for systems using DataFusion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100