influxdata / influxdata/influxdb

batch telegraf data after disconnect leaves holes in continuous query tables

Open
#19,698 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.

1. setup telegraf on host to send data
[agent]
interval = "60s"
round_interval = true
metric_batch_size = 1000 // allow batched data
metric_buffer_limit = 10000 // allow sufficient data aggregation over disconnect period
...
2. setup second host with influx and continuous query
DROP CONTINUOUS QUERY "cq.starts" ON "app";
CREATE CONTINUOUS QUERY "cq.starts" ON "app"
BEGIN
SELECT
non_negative_difference(max(global_starts))
as starts,
INTO "app"."one-year"."summary.starts"
FROM "app"."autogen"."data.starts"
GROUP BY time(1m), * fill(none)
END;

3. take down host running influx for a period of time (say 15minutes) and restart
4. view graphs using the continuous query table and note 15minute gap of data

__Expected behavior:__
Describe what you expected to happen.
The continuous query should process the delayed batch data and fill in the hole created during the disconnected interval when the system is re-connected.

__Actual behavior:__
Describe What actually happened.
The continuous query data is left blank for the interval.

__Environment info:__

* System info: Run `` and copy the output here
inux influxdb-5786ccb9f8-pjmqj 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 Linux
* InfluxDB version: Run `influxd version` and copy the output here
InfluxDB v1.7.8 (git: 1.7 ff383cdc0420217e3460dabe17db54f8557d95b6)
* Other relevant environment details: Container runtime, disk info, etc

__Config:__
Copy any non-default config values here or attach the full config as a gist or file.

__Logs:__
Include snippet of errors in log.

__Performance:__
Generate profiles with the following commands for bugs related to performance, locking, out of memory (OOM), etc.

```sh
# Commands should be run when the bug is actively happening.
# Note: This command will run for at least 30 seconds.
curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=true"
curl -o vars.txt "http://localhost:8086/debug/vars"
iostat -xd 1 30 > iostat.txt
# Attach the `profiles.tar.gz`, `vars.txt`, and `iostat.txt` output files.
```

Contributor guide

Open the contributing guide

Research direction

No source files or tests are named. Start by reproducing the disconnect and delayed Telegraf batch with the provided continuous query, then trace the continuous-query processing for the missing interval. Done means delayed data fills the gap in the continuous-query table after InfluxDB restarts.

Written by the indexing model from the issue text.

Assessment

Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.