influxdata / influxdata/influxdb

Error: failed to write data: 500 Internal Server Error: unexpected error writing points to database: [shard 960]

Open
#23,438 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

Related to https://github.com/influxdata/influxdb/issues/23212

__Steps to reproduce:__

1. On a populated server (2.0.5) run: influxd inspect export-lp --bucket-id 540a390190697254 --engine-path /ssd/influx/engine --output-path /ssd/influx/backups/export.lp --start 2022-01-01T00:00:00Z --end 2022-01-02T23:59:59Z
2. On another server (2.2) run: influx write --bucket home_assistant --file ~/export.lp

__Expected behavior:__
Data appears in the new server

__Actual behavior:__
Error: failed to write data: 500 Internal Server Error: unexpected error writing points to database: [shard 960] open /home/pi/.influxdbv2/engine/data/514933d6df66d74a/autogen/960/index/1/MANIFEST: too many open files

__Environment info:__

* System info: Linux 5.10.103-v8+ aarch64 GNU/Linux (Bullseye)
* From: InfluxDB version: InfluxDB 2.0.5 (git: https://github.com/influxdata/influxdb/commit/741389781e88ecf1f28ff34aa04aa5cf22d99ae4) build_date: 2021-04-27T17:57:07Z
* To: InfluxDB v2.2.0 (git: a2f8538837) build_date: 2022-04-06T17:36:38Z
* Other relevant environment details: Raspberry Pi3 with fresh OS, updated & upgraded, no other modifications made/programs installed.

Contributor guide

Open the contributing guide

Research direction

Start with the `influxd inspect export-lp` and `influx write` entry points described in the reproduction steps, then reproduce the import on Linux while monitoring open files. Investigate why importing the exported data reaches the shard manifest limit; done means the same export imports successfully without the “too many open files” error.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux
Domain
cli, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.