influxdata / influxdata/influxdb
Does docker InfluxDB v1.8 support collectD with security-level = "encrypt" ?
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
Nothing I see gives me confidence that it does. I read some old reports here. [PR](https://github.com/influxdata/influxdb/pull/7435) added functionality from v1.2 - Check.
I run `docker pull influxdb` for the `:latest` tag, and it only pulls in 1.8.1 when release 1.8.2 says it should be latest...
Docs:
https://docs.influxdata.com/influxdb/v1.8/supported_protocols/collectd/
` security-level = "none" # "none", "sign", or "encrypt"`
BUT
https://docs.influxdata.com/influxdb/v1.8/administration/config/#collectd-settings
says:
`security-level = "none"`
but
https://docs.influxdata.com/enterprise_influxdb/v1.8/administration/config-data-nodes#collectd-settings
says:
```
security-level = ""
The collectd security level can be “” (or “none”), “sign”, or “encrypt”.
```
Enterprise only feature????
I run docker with my env.file so:
```
INFLUXDB_DATA_ENGINE=tsm1
INFLUXDB_REPORTING_DISABLED=false
INFLUXDB_LOGGING_LEVEL=debug
INFLUXDB_COLLECTD_ENABLED=true
INFLUXDB_COLLECTD_DATABASE=collectd
INFLUXDB_COLLECTD_BIND_ADDRESS=:25826
INFLUXDB_COLLECTD_SECURITY_LEVEL=encrypt
INFLUXDB_COLLECTD_AUTH_FILE=/etc/collectd/auth_file
INFLUXDB_COLLECTD_TYPESDB=/usr/share/collectd/types.db
```
and my `auth_file` appropriately set for:
```
user: password
```
but when I run `docker-compose exec influxdb sh` and look around, the exports are there:
```bash
# set
HOME='/root'
HOSTNAME='influxdb'
IFS='
'
INFLUXDB_COLLECTD_AUTH_FILE='/etc/collectd/auth_file'
INFLUXDB_COLLECTD_BIND_ADDRESS=':25826'
INFLUXDB_COLLECTD_DATABASE='collectd'
INFLUXDB_COLLECTD_ENABLED='true'
INFLUXDB_COLLECTD_SECURITY_LEVEL='encrypt'
INFLUXDB_COLLECTD_TYPESDB='/usr/share/collectd/types.db'
INFLUXDB_DATA_ENGINE='tsm1'
INFLUXDB_LOGGING_LEVEL='debug'
INFLUXDB_REPORTING_DISABLED='false'
INFLUXDB_VERSION='1.8.1'
OPTIND='1'
PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
PPID='0'
PS1='# '
PS2='> '
PS4='+ '
PWD='/'
TERM='xterm'
```
but the config doesn't seem to reflect the settings:
```bash
# cat /etc/influxdb/influxdb.conf
[meta]
dir = "/var/lib/influxdb/meta"
[data]
dir = "/var/lib/influxdb/data"
engine = "tsm1"
wal-dir = "/var/lib/influxdb/wal"
```
If I set `INFLUXDB_COLLECTD_SECURITY_LEVEL=Encrypt` the docker image wont start, and reports the error
```
run: invalid collectd config: Invalid security level. To generate a valid configuration file run `influxd config > influxdb.generated.conf`
```
But if I set `INFLUXDB_COLLECTD_SECURITY_LEVEL=encrypt` it starts, but no indication or log messages that it takes this:
```
ts=2020-08-19T13:58:13.756871Z lvl=info msg="InfluxDB starting" log_id=0Oil2w~0000 version=1.8.1 branch=1.8 commit=af0237819ab9c5997c1c0144862dc762b9d8fc25
ts=2020-08-19T13:58:13.756957Z lvl=info msg="Go runtime" log_id=0Oil2w~0000 version=go1.13.8 maxprocs=4
ts=2020-08-19T13:58:13.858966Z lvl=info msg="Using data dir" log_id=0Oil2w~0000 service=store path=/var/lib/influxdb/data
ts=2020-08-19T13:58:13.859028Z lvl=info msg="Compaction settings" log_id=0Oil2w~0000 service=store max_concurrent_compactions=2 throughput_bytes_per_second=50331648 throughput_bytes_per_second_burst=50331648
ts=2020-08-19T13:58:13.859051Z lvl=info msg="Open store (start)" log_id=0Oil2w~0000 service=store trace_id=0Oil2xOl000 op_name=tsdb_open op_event=start
ts=2020-08-19T13:58:13.888044Z lvl=info msg="Reading file" log_id=0Oil2w~0000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/_internal/monitor/2/_00001.wal size=10486175
ts=2020-08-19T13:58:13.888313Z lvl=info msg="Opened file" log_id=0Oil2w~0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/1/000000001-000000001.tsm id=0 duration=0.187ms
ts=2020-08-19T13:58:13.967251Z lvl=info msg="Opened shard" log_id=0Oil2w~0000 service=store trace_id=0Oil2xOl000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/_internal/monitor/1 duration=80.203ms
ts=2020-08-19T13:58:14.734679Z lvl=info msg="Reading file" log_id=0Oil2w~0000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/_internal/monitor/2/_00002.wal size=8533262
ts=2020-08-19T13:58:15.503957Z lvl=info msg="Opened shard" log_id=0Oil2w~0000 service=store trace_id=0Oil2xOl000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/_internal/monitor/2 duration=1616.972ms
ts=2020-08-19T13:58:15.504319Z lvl=info msg="Open store (end)" log_id=0Oil2w~0000 service=store trace_id=0Oil2xOl000 op_name=tsdb_open op_event=end op_elapsed=1645.265ms
ts=2020-08-19T13:58:15.504401Z lvl=info msg="Opened service" log_id=0Oil2w~0000 service=subscriber
ts=2020-08-19T13:58:15.504420Z lvl=info msg="Starting monitor service" log_id=0Oil2w~0000 service=monitor
ts=2020-08-19T13:58:15.504432Z lvl=info msg="Registered diagnostics client" log_id=0Oil2w~0000 service=monitor name=build
ts=2020-08-19T13:58:15.504441Z lvl=info msg="Registered diagnostics client" log_id=0Oil2w~0000 service=monitor name=runtime
ts=2020-08-19T13:58:15.504455Z lvl=info msg="Registered diagnostics client" log_id=0Oil2w~0000 service=monitor name=network
ts=2020-08-19T13:58:15.504464Z lvl=info msg="Registered diagnostics client" log_id=0Oil2w~0000 service=monitor name=system
ts=2020-08-19T13:58:15.504503Z lvl=info msg="Starting precreation service" log_id=0Oil2w~0000 service=shard-precreation check_interval=10m advance_period=30m
ts=2020-08-19T13:58:15.504522Z lvl=info msg="Starting snapshot service" log_id=0Oil2w~0000 service=snapshot
ts=2020-08-19T13:58:15.504533Z lvl=info msg="Starting continuous query service" log_id=0Oil2w~0000 service=continuous_querier
ts=2020-08-19T13:58:15.504545Z lvl=info msg="Starting HTTP service" log_id=0Oil2w~0000 service=httpd authentication=false
ts=2020-08-19T13:58:15.504555Z lvl=info msg="opened HTTP access log" log_id=0Oil2w~0000 service=httpd path=stderr
ts=2020-08-19T13:58:15.504564Z lvl=info msg="Storing statistics" log_id=0Oil2w~0000 service=monitor db_instance=_internal db_rp=monitor interval=10s
ts=2020-08-19T13:58:15.504748Z lvl=info msg="Listening on HTTP" log_id=0Oil2w~0000 service=httpd addr=[::]:8086 https=false
ts=2020-08-19T13:58:15.504802Z lvl=info msg="Starting retention policy enforcement service" log_id=0Oil2w~0000 service=retention check_interval=30m
ts=2020-08-19T13:58:15.504860Z lvl=info msg="Starting collectd service" log_id=0Oil2w~0000 service=collectd
ts=2020-08-19T13:58:15.505105Z lvl=info msg="Loading types from file" log_id=0Oil2w~0000 service=collectd path=/usr/share/collectd/types.db
ts=2020-08-19T13:58:15.506811Z lvl=info msg="Listening on UDP" log_id=0Oil2w~0000 service=collectd addr=[::]:25826
ts=2020-08-19T13:58:15.507277Z lvl=info msg="Listening for signals" log_id=0Oil2w~0000
ts=2020-08-19T13:58:15.508046Z lvl=info msg="Sending usage statistics to usage.influxdata.com" log_id=0Oil2w~0000
```
The docker image file `init-influxdb.sh` gives me no confidence that the settings are ever read.
Does the influxd or some other Daemon read the environment variables directly?
output from :8086/debug/vars :
```
...
"collectd::25826": {"name":"collectd","tags":{"bind":":25826"},"values":{"batchesTx":0,"batchesTxFail":0,"bytesRx":0,"droppedPointsInvalid":0,"pointsParseFail":0,"pointsRx":0,"pointsTx":0,"readFail":0}}
...
```
tcpdump shows encrypted packets ingressing to my docker image - but the DB remains empty.
I even tried setting `INFLUXDB_LOGGING_LEVEL=debug` but it does nothing evident in the docker image.
It seems to me that the docker image is neutered and can't do anything with most of the environment variables documented.
__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.
1. Configured two docker images, both with collectd. One with encrypt. One without.
2. Docker without encrypt: db fills. Docker with encrypt. db stays empty.
3. help?
__Expected behavior:__
influxdb to adhere to the passed environment variables documented everywhere in the docs.
__Actual behavior:__
Nada.
__Environment info:__
Docker - latest.
* System info: Linux 3.10.0-1127.18.2.el7.x86_64 x86_64
* InfluxDB version: InfluxDB v1.8.1 (git: 1.8 af0237819ab9c5997c1c0144862dc762b9d8fc25)
__Config:__
Copy any non-default config values here or attach the full config as a gist or file.
__Logs:__
No errors. Image does not observe the debug logging environment flag passed.
__Performance:__
Pointless.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Docker image's init-influxdb.sh and the generated /etc/influxdb/influxdb.conf, then compare the documented collectd settings with the environment variables shown in the issue. Reproduce the encrypt and non-encrypt cases and inspect :8086/debug/vars; done means determining whether the variables are applied and whether encrypted collectd packets are accepted.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker
- Domain
- database, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100