influxdata / influxdata/influxdb
influxd service stops working until a reboot on local macOS server
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
__Steps to reproduce:__
At the moment I can't give an exact way to reproduce this. `influxd` service stops running every day-ish or so.
__Expected behavior:__
InfluxDB to run actively and `influxd` service to not randomly stop working. The only way to restore service is to reboot the machine.
__Actual behavior:__
I have a 2011 Mac mini set up as a local server that parses data through a Python script and appends it to several databases. This is done every ten minutes through a cronjob and works without a problem. I need this operations to be kinda reliable, don't mind if every so once in a while the script fails and data is not added to the database but it needs to be consistent. As I am using a home server I am using a [Healthchecks.io](https://healthchecks.io) to monitor whenever a cron job fails. Every so often, can't specify why or how it happens I get a notification through the previous service that the script failed. I thought a while ago that it could be to an internet loss connection problem so I wired it to the router in hopes of it being fixed but I've come to realize today that the `influxd` service stops working.
Whenever that happens I SSH into the machine and running the following gives more or less an indication on what happened.
```
> influx
Failed to connect to http://localhost:8086: Get http://localhost:8086/ping: dial tcp [::1]:8086: connect: connection refused
Please check your connection settings and ensure `influxd` is running.
```
The way I run this kind fo script is using a cronjob like this line shows:
```
*/10 * * * * /usr/local/bin/python3 /Users/javierdemartin/Documents/neural-bikes-parsers/parsers/paris.py && curl -fsS --retry 3 https://hc-ping.com/MY_KEY
```
A sample of the mentioned Python script is accesible [here](https://github.com/javierdemartin/neural-bikes-parsers/blob/master/parsers/bilbao.py). I am certain the issue is not there but just you never know.
__Environment info:__
Installed on a Mac mini through brew
```
> uname -srm
Darwin 17.7.0 x86_64
```
```
> influxd version
InfluxDB vv1.7.9 (git: master 23bc63d43a8dc05f53afa46e3526ebb5578f3d88)
```
__Config:__
Copy any non-default config values here or attach the full config as a gist or file.
__Logs:__
```
> influx
Failed to connect to http://localhost:8086: Get http://localhost:8086/ping: dial tcp [::1]:8086: connect: connection refused
Please check your connection settings and ensure `influxd` is running.
```
__Performance:__
I am not able to run the following whenever the bug happens as `influxd` stops working so I don't have access to `localhost:8086`.
```sh
# Commands should be run when the bug is actively happening.
# Note: This command will run for at least 30 seconds.
curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=true"
curl -o vars.txt "http://localhost:8086/debug/vars"
iostat -xd 1 30 > iostat.txt
# Attach the `profiles.tar.gz`, `vars.txt`, and `iostat.txt` output files.
```
Contributor guide
Research direction
Start by investigating the `influxd` service on the reported macOS environment when `localhost:8086` refuses connections, using the `influx` failure and the provided service context. Reproduce the daily stop or collect relevant service logs and the listed pprof, vars, and iostat outputs; done means identifying the cause and preventing the service from stopping without a reboot.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos, python
- Domain
- databases, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100