apache / apache/arrow-rs-object-store
[object_store] API to calculate memory consumption of `ObjectMeta`
- Dominant language
- Rust
- Stars
- 322
- Forks
- 212
- Avg merge
- 5d 2h
- Merged PRs (30d)
- 10
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
In InfluxDB IOx we cache `ObjectMeta` (among other things) and thus calculate the memory used in order to limit the amount of memory used for caching
**Describe the solution you'd like**
In order to have accurate memory usage accounting, it would be nice to have an API like
```rust
impl ObjectMeta {
/// return the total bytes of memory used by this object, including heap space
pub fn memory_bytes(&self) -> usize {
...
}
}
```
**Describe alternatives you've considered**
We could cache the raw data directly as `Strings` and recreate `ObjectMeta` when needed
**Additional context**
See context in https://github.com/influxdata/influxdb_iox/pull/8577/files#r1316378812
Contributor guide
Research direction
Start by locating the Rust definition of ObjectMeta and reviewing how its stored data contributes to memory use. Add the requested memory_bytes API, with completion defined as reporting the total bytes used by an ObjectMeta instance, including heap space.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design, performance
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100