influxdata / influxdata/influxdb
InfluxDB API at http://localhost:8086/ready unavailable perpetually
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
We built an influx v2.7 on Ubuntu with default settings. The only thing we did was point the engine to a new drive.
This database worked fine for weeks and we even loaded it with 12 TB of data. However, it was left alone for a few weeks and when the users came back, the DB was unresponsive.
Now, no matter what we do, we cannot get the service to start successfully and the logs just show that InfluxDB API at http://localhost:8086/ready unavailable ....
We tried adjusting the config file and beefing it up past the defaults.
We tried commenting out Type=forking in the systemd service file
We confirmed that the settings seem to match out other working instances
Restarting the machine and the service
We tried a lot of odds an and ends and cant seem to get it to start
__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.
1. deploy influxdb2 on Ubuntu with default config
2. use it, load 12 TB data, leave it
3. upon unresponsiveness, try to restart it to get the service active
__Expected behavior:__
Influx to work normally
__Actual behavior:__
Influx service and database will not start or become responsive
__Environment info:__
* System info: Linux 5.15.0-78-generic x86_64
* InfluxDB version: InfluxDB v2.7.1 (git: 407fa622e9) build_date: 2023-04-28T13:24:27Z
* Other relevant environment details: ubuntu VM on vsphere; bolt on standard drive (100 GB) and engine on separate drive (15 TB); 64 vCPU; 256 GB RAM.
__Config:__
Original config we had when the break happened:
`cat /etc/influxdb/config.toml`
```
bolt-path = "/var/lib/influxdb/influxd.bolt"
engine-path = "/mnt/data/influxdb/engine"
```
Updated config for more resources, still not working:
`cat /etc/influxdb/config.toml`
```
bolt-path = "/var/lib/influxdb/influxd.bolt"
engine-path = "/mnt/data/influxdb/engine"
http-bind-address = ":8086"
tracing-type = "log"
storage-cache-max-memory-size = 107374182400
storage-max-concurrent-compactions = 32
storage-compact-throughput-burst = 1073741824
query-concurrency = 64
query-queue-size = 100
query-initial-memory-bytes = 2147483648
query-max-memory-bytes = 137438953472
session-length = 120
```
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the systemd service status and logs for the InfluxDB v2.7.1 instance, then compare the reported config paths and storage layout with a working instance. The issue does not name a source file or test and provides no startup log details, so the failure and a verifiable fix still need to be narrowed down.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- linux, ubuntu
- Domain
- databases, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100