influxdata / influxdata/influxdb

(1.7.8) Massive RAM exhaustion when using SELECT INTO in restore process

Open
#15,560 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

Hi,

Using version 1.7.8 on CentOS 7, systems with 256GB RAM.

On instance A I have performed a portable shards backup on instance A, for multiple databases. Shard duration is one day, a single shard was backed up. The backup for one database is around 1GB.
On instance B, I performed restore using -newdb setting. Then started using SELECT INTO to restore missing points. For small databases this works fine. The 1GB shard backup restored into a 3.2GB database. The destination database has quite some data accumulated, and all shards in total are occupying ~800GB on the filesystem. Daily shard's are between 3 and 5 GB.

When I do the SELECT INTO, InfluxDB climbs from ~64GB up to the server's 256GB, then OOM happens.

This happens even when trying to SELECT only 1 hour worth of points, as such:
SELECT * INTO telegraf..:MEASUREMENT FROM /.*/ WHERE time > '2019-10-22T00:00:00Z' AND time < '2019-10-22T01:00:00Z' GROUP BY *

Appreciate some help in understanding how this happens this way (bug, design, something I should be doing differently?)

Regards.

Contributor guide

Open the contributing guide

Research direction

Reproduce the restore workflow using -newdb and the reported SELECT INTO query on InfluxDB 1.7.8, starting with the one-hour time range and monitoring memory usage. Compare small and accumulated databases to identify whether the OOM is expected behavior or a defect; done means documenting the cause and a verified resolution.

Written by the indexing model from the issue text.

Assessment

Domain
databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.