influxdata / influxdata/influxdb

Using mmap instead of allocating large buffer in memory

Open
#23,711 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

https://github.com/influxdata/influxdb/blob/df6a5e68e0e61a638b24a0779dc54c51667d52fa/tsdb/series_partition.go#L582

Hello, I am using InfluxDB 1.x and I find when series cardinality is to high, we need allocate too much memory here to compact series. Why don't we use mmap here to avoid large memory allocation?

Contributor guide

Open the contributing guide

Research direction

Start with tsdb/series_partition.go around line 582, as linked in the issue, and trace how compaction handles high series cardinality and its buffer allocation. Determine whether replacing that allocation with mmap is feasible for InfluxDB 1.x, then document or implement the approach with coverage for the affected compaction behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.