influxdata / influxdata/influxdb
Default unix socket fails to initialize on modern Linux installations
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.
1. Login to an Ubuntu or Armbian Linux box
2. Set unix-socket-enabled to true
3. Restart influxdb
__Expected behavior:__
I expected influxdb to start with and listening to the socket
I expected the socket to be opened r/w for user and group, not r/w for others.
__Actual behavior:__
The unix server socket could not be initialized, since /var/run is not writable by the influxdb user for security reasons. The rest of influxdb functionally also stops, the the startup-wrapper keeps retrying to connect.
When a socket is opened, it is opened r/w for user, group and others. It would be more secure to disable write for others, so the access to the socket can be controlled by assigning influxdb users to /etc/group.
Normal practice is to use a subfolder in /var/run, e.g. /var/run/influxdb and have that owned by user:group influxdb:influxdb as part of the startup wrapper (systemd or init.d)
When I configure to use this folder, the socket is opened with r/w for others which gives all local users access to the socket:
```
% ls -lsa /var/run/influxdb
total 0
0 drwxr-xr-x 2 influxdb influxdb 60 Aug 16 09:54 .
0 drwxr-xr-x 27 root root 800 Aug 16 09:33 ..
0 srwxrwxrwx 1 influxdb influxdb 0 Aug 16 09:54 influxdb.sock
```
The /var/run/influxdb.sock appears to be in the default configuration as well as in the [code](/influxdata/influxdb/blob/1.8.4/services/httpd/config.go#L24)
__Environment info:__
* System info: Run `uname -srm` and copy the output here
```
% uname -srm
Linux 5.10.63-rockchip64 aarch64
```
* InfluxDB version: Run `influxd version` and copy the output here
```
% influxd version
InfluxDB v1.8.10 (git: 1.8 688e697c51fd)
```
* Other relevant environment details: Container runtime, disk info, etc
On Armbian Linux:
```
% ls -ld /var/run/
drwxr-xr-x 27 root root 800 Aug 16 09:33 /var/run/
```
On Ubuntu:
```
$ ls -ld /var/run/
drwxr-xr-x 52 root root 1720 Aug 16 06:56 /var/run/
```
__Config:__
Copy any non-default config values here or attach the full config as a gist or file.
influxdb.conf, defaults start with #
```
# Enable http service over unix domain socket
# unix-socket-enabled = false
unix-socket-enabled = true
# The path of the unix domain socket.
# bind-socket = "/var/run/influxdb.sock"
```
__Logs:__
```
Aug 16 07:13:38 bullseyeTestCT influxd-systemd-start.sh[205724]: ts=2023-08-16T07:13:38.950565Z lvl=info msg="Starting HTTP service" log_id=0jgIvj6G000 service=httpd authentication=false
Aug 16 07:13:38 bullseyeTestCT influxd-systemd-start.sh[205724]: ts=2023-08-16T07:13:38.951209Z lvl=info msg="Listening on HTTP" log_id=0jgIvj6G000 service=httpd addr=[::]:8086 https=false
Aug 16 07:13:38 bullseyeTestCT influxd-systemd-start.sh[205724]: run: open server: open service: listen unix /var/run/influxdb.sock: bind: permission denied
Aug 16 07:13:39 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 6 attempts...
Aug 16 07:13:40 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 7 attempts...
Aug 16 07:13:41 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 8 attempts...
Aug 16 07:13:42 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 9 attempts...
Aug 16 07:13:43 bullseyeTestCT telegraf[1368]: 2023-08-16T07:13:43Z E! [outputs.influxdb] When writing to [http://localhost:8086]: failed doing req: Post "http://localhost:8086/write?db=telegraf": dial tcp [::1]:8086: connect: connection refused
Aug 16 07:13:43 bullseyeTestCT telegraf[1368]: 2023-08-16T07:13:43Z E! [agent] Error writing to outputs.influxdb: could not write any address
Aug 16 07:13:43 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 10 attempts...
Aug 16 07:13:44 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 11 attempts...
Aug 16 07:13:45 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 12 attempts...
Aug 16 07:13:46 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 13 attempts...
Aug 16 07:13:47 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 14 attempts...
Aug 16 07:13:48 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 15 attempts...
Aug 16 07:13:50 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 16 attempts...
Aug 16 07:13:51 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 17 attempts...
Aug 16 07:13:52 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 18 attempts...
Aug 16 07:13:53 bullseyeTestCT telegraf[1368]: 2023-08-16T07:13:53Z E! [outputs.influxdb] When writing to [http://localhost:8086]: failed doing req: Post "http://localhost:8086/write?db=telegraf": dial tcp [::1]:8086: connect: connection refused
Aug 16 07:13:53 bullseyeTestCT telegraf[1368]: 2023-08-16T07:13:53Z E! [agent] Error writing to outputs.influxdb: could not write any address
Aug 16 07:13:54 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 20 attempts...
Aug 16 07:13:55 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 21 attempts...
Aug 16 07:13:56 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 22 attempts...
Aug 16 07:13:57 bullseyeTestCT influxd-systemd-start.sh[205723]: InfluxDB API unavailable after 23 attempts...
$ sudo systemctl restart influxdb.service
Aug 16 07:13:57 bullseyeTestCT sudo[205851]: dennis : TTY=pts/0 ; PWD=/home/dennis ; USER=root ; COMMAND=/usr/bin/systemctl restart influxdb.service
Aug 16 07:13:57 bullseyeTestCT sudo[205851]: pam_unix(sudo:session): session opened for user root(uid=0) by dennis(uid=1000)
Aug 16 07:13:57 bullseyeTestCT systemd[1]: influxdb.service: Succeeded.
Aug 16 07:13:57 bullseyeTestCT systemd[1]: Stopped InfluxDB is an open-source, distributed, time series database.
Aug 16 07:13:57 bullseyeTestCT systemd[1]: influxdb.service: Consumed 9.592s CPU time.
Aug 16 07:13:57 bullseyeTestCT systemd[1]: Starting InfluxDB is an open-source, distributed, time series database...
Aug 16 07:13:57 bullseyeTestCT influxd-systemd-start.sh[205855]: ts=2023-08-16T07:13:57.949734Z lvl=info msg="InfluxDB starting" log_id=0jgIxEkG000 version=1.8.10 branch=1.8 commit=688e697c51fd
Aug 16 07:13:57 bullseyeTestCT influxd-systemd-start.sh[205855]: ts=2023-08-16T07:13:57.949992Z lvl=info msg="Go runtime" log_id=0jgIxEkG000 version=go1.13.8 maxprocs=2
Aug 16 07:13:57 bullseyeTestCT influxd-systemd-start.sh[205857]: Merging with configuration at: /etc/influxdb/influxdb.conf
Aug 16 07:13:58 bullseyeTestCT influxd-systemd-start.sh[205855]: ts=2023-08-16T07:13:58.056049Z lvl=info msg="Using data dir" log_id=0jgIxEkG000 service=store path=/var/lib/influxdb/data
Aug 16 07:13:58 bullseyeTestCT influxd-systemd-start.sh[205855]: ts=2023-08-16T07:13:58.056273Z lvl=info msg="Compaction settings" log_id=0jgIxEkG000 service=store max_concurrent_compactions=1 throughput_bytes_per_second=1048576 throughput_bytes_per_second_burst=10485760
Aug 16 07:13:58 bullseyeTestCT influxd-systemd-start.sh[205855]: ts=2023-08-16T07:13:58.056401Z lvl=info msg="Open store (start)" log_id=0jgIxEkG000 service=store trace_id=0jgIxFA0000 op_name=tsdb_open op_event=start
Aug 16 07:13:58 bullseyeTestCT influxd-systemd-start.sh[205871]: Merging with configuration at: /etc/influxdb/influxdb.conf
Aug 16 07:13:58 bullseyeTestCT influxd-systemd-start.sh[205854]: InfluxDB API unavailable after 1 attempts...
... snapshots e.g.
Aug 16 07:14:03 bullseyeTestCT influxd-systemd-start.sh[205855]: ts=2023-08-16T07:14:03.994739Z lvl=info msg="Starting snapshot service" log_id=0jgIxEkG000 service=snapshot
Aug 16 07:14:03 bullseyeTestCT influxd-systemd-start.sh[205855]: ts=2023-08-16T07:14:03.995485Z lvl=info msg="Starting continuous query service" log_id=0jgIxEkG000 service=continuous_querier
Aug 16 07:14:03 bullseyeTestCT influxd-systemd-start.sh[205855]: ts=2023-08-16T07:14:03.995772Z lvl=info msg="Starting HTTP service" log_id=0jgIxEkG000 service=httpd authentication=false
Aug 16 07:14:03 bullseyeTestCT influxd-systemd-start.sh[205855]: ts=2023-08-16T07:14:03.996670Z lvl=info msg="Listening on HTTP" log_id=0jgIxEkG000 service=httpd addr=[::]:8086 https=false
Aug 16 07:14:04 bullseyeTestCT influxd-systemd-start.sh[205854]: InfluxDB API unavailable after 6 attempts...
Aug 16 07:14:05 bullseyeTestCT influxd-systemd-start.sh[205854]: InfluxDB API unavailable after 7 attempts...
Aug 16 07:14:06 bullseyeTestCT influxd-systemd-start.sh[205854]: InfluxDB API unavailable after 8 attempts...
Aug 16 07:14:07 bullseyeTestCT influxd-systemd-start.sh[205854]: InfluxDB API unavailable after 9 attempts...
Aug 16 07:14:08 bullseyeTestCT influxd-systemd-start.sh[205854]: InfluxDB API unavailable after 10 attempts...
Aug 16 07:14:09 bullseyeTestCT influxd-systemd-start.sh[205854]: InfluxDB API unavailable after 11 attempts...
Aug 16 07:14:10 bullseyeTestCT influxd-systemd-start.sh[205854]: InfluxDB API unavailable after 12 attempts...
...
Aug 16 07:14:04 bullseyeTestCT influxd-systemd-start.sh[205855]: run: open server: open service: listen unix /var/run/influxdb.sock: bind: permission denied
Aug 16 07:15:34 bullseyeTestCT influxd-systemd-start.sh[206227]: run: open server: open service: listen unix /var/run/influxdb.sock: bind: permission denied
```
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with services/httpd/config.go, where the default Unix socket path is referenced, then inspect the systemd or init.d startup wrapper involved in launching InfluxDB. Reproduce with unix-socket-enabled=true on Ubuntu or Armbian and verify that startup succeeds and the socket is accessible to the intended user and group without write access for others.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, linux
- Domain
- backend, databases, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100