influxdata / influxdata/influxdb

Questions about startup behavior and hot/cold data rules with TSI index in InfluxDB 1.x

Open
#27,600 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

Background:
I'm using InfluxDB 1.x with TSI (tsi1) index enabled.

Observation:
When the database starts up, it scans through all TSM files. This seems to take a significant amount of time, especially with a large dataset.

Core Question:
According to the official documentation, TSI leverages the operating system's page cache to manage hot data, keeping frequently accessed index data in memory while cold data remains on disk. However, I couldn't find a clear explanation of how "hot" and "cold" data are actually determined.
How can I improve InfluxDB's startup speed when there are a large number of shards?

Contributor guide

Open the contributing guide

Research direction

Start with the official InfluxDB 1.x documentation for TSI, then trace the startup scan behavior for TSM files and multiple shards. Document how hot and cold index data are determined and identify supported ways to improve startup time; the issue does not name a documentation file or test.

Written by the indexing model from the issue text.

Assessment

Domain
databases
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.