influxdata / influxdata/influxdb
Using mmap instead of allocating large buffer in memory
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
https://github.com/influxdata/influxdb/blob/df6a5e68e0e61a638b24a0779dc54c51667d52fa/tsdb/series_partition.go#L582
Hello, I am using InfluxDB 1.x and I find when series cardinality is to high, we need allocate too much memory here to compact series. Why don't we use mmap here to avoid large memory allocation?
Contributor guide
Research direction
Start with tsdb/series_partition.go around line 582, as linked in the issue, and trace how compaction handles high series cardinality and its buffer allocation. Determine whether replacing that allocation with mmap is feasible for InfluxDB 1.x, then document or implement the approach with coverage for the affected compaction behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100