influxdata / influxdata/influxdb
batch telegraf data after disconnect leaves holes in continuous query tables
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.
1. setup telegraf on host to send data
[agent]
interval = "60s"
round_interval = true
metric_batch_size = 1000 // allow batched data
metric_buffer_limit = 10000 // allow sufficient data aggregation over disconnect period
...
2. setup second host with influx and continuous query
DROP CONTINUOUS QUERY "cq.starts" ON "app";
CREATE CONTINUOUS QUERY "cq.starts" ON "app"
BEGIN
SELECT
non_negative_difference(max(global_starts))
as starts,
INTO "app"."one-year"."summary.starts"
FROM "app"."autogen"."data.starts"
GROUP BY time(1m), * fill(none)
END;
3. take down host running influx for a period of time (say 15minutes) and restart
4. view graphs using the continuous query table and note 15minute gap of data
__Expected behavior:__
Describe what you expected to happen.
The continuous query should process the delayed batch data and fill in the hole created during the disconnected interval when the system is re-connected.
__Actual behavior:__
Describe What actually happened.
The continuous query data is left blank for the interval.
__Environment info:__
* System info: Run `` and copy the output here
inux influxdb-5786ccb9f8-pjmqj 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 Linux
* InfluxDB version: Run `influxd version` and copy the output here
InfluxDB v1.7.8 (git: 1.7 ff383cdc0420217e3460dabe17db54f8557d95b6)
* Other relevant environment details: Container runtime, disk info, etc
__Config:__
Copy any non-default config values here or attach the full config as a gist or file.
__Logs:__
Include snippet of errors in log.
__Performance:__
Generate profiles with the following commands for bugs related to performance, locking, out of memory (OOM), etc.
```sh
# Commands should be run when the bug is actively happening.
# Note: This command will run for at least 30 seconds.
curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=true"
curl -o vars.txt "http://localhost:8086/debug/vars"
iostat -xd 1 30 > iostat.txt
# Attach the `profiles.tar.gz`, `vars.txt`, and `iostat.txt` output files.
```
Contributor guide
Research direction
No source files or tests are named. Start by reproducing the disconnect and delayed Telegraf batch with the provided continuous query, then trace the continuous-query processing for the missing interval. Done means delayed data fills the gap in the continuous-query table after InfluxDB restarts.
Written by the indexing model from the issue text.
Assessment
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100