influxdata / influxdata/influxdb

InfluxDB2 Task - First Regex Match Being Missed (Tasks)

Open
#20,792 1 comment 0 reactions 0 assignees View on GitHub
area/2.x area/flux area/queries area/tasks
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.

1. Apply/Utilise Default Configuration for Telegraf Windows
2. Import Task to Downsample Data (Downsample All) as per https://github.com/influxdata/community-templates/tree/master/downsampling
3. Execute Task i.e. "downsample-all-10m" against specified Source (From Bucket) and Destination (To Bucket)

__Expected behavior:__
All fields where there is a partial match using the supplied regex should be caught and downsampled.

__Actual behavior:__
The first field in the regex partial match is not executed by the task
This can be demonstrated by either:

1. Rearranging the partial regex to demonstrate that the match occurs successfully as long as it is not first. However, whatever is now first will be missed further proving the issue.
2. Entering a random unique string to effectively be caught first and allow all others to be matched works but is non-desirable

i.e.:
all_data = from(bucket: fromBucket)
|> range(start: -task.every)
|> filter(fn: (r) =>
(r._measurement =~ /^cpu|disk|diskio|mem|processes|swap|system|internal_.+|net|netstat|procstat|procstat_lookup|smart_.+$/))

Using the supplied downsample above, any measurement which contains "cpu" is missed. However, using either of the above options of rearranging the regex so that "cpu" isn't first or entering a dummy value at the start will enable "cpu" to match successfully.

It appears for whatever reason, InfluxDB2 is missing the first statement in the regex where measurement is assigned and then executed. As long as it is not the first measurement, it is downsampled correctly.

__Environment info:__

* System info: docker exec -ti influxdb uname -srm
Linux 5.4.0-64-generic x86_64

* InfluxDB version: docker exec -ti influxdb influxd
2021-02-22T11:55:29.944943Z info Welcome to InfluxDB {"log_id": "0SUPMJ60000", "version": "2.0.4", "commit": "4e7a59bb9a", "build_date": "2021-02-08T17:47:02Z"}
2021-02-22T11:55:30.901368Z error Failed opening bolt {"log_id": "0SUPMJ60000", "error": "unable to open boltdb: timeout"}
Error: unable to open boltdb: timeout

_Not sure why the error related to boltdb occurs._

* Other relevant environment details:
InfluxDB Docker Container - quay.io/influxdb/influxdb

__Config:__

Telegraf Config - https://github.com/influxdata/telegraf/blob/master/etc/telegraf.conf
Downsample Config (I've utilised downsample-all-10m) - https://github.com/influxdata/community-templates/blob/master/downsampling/all_inputs/downsampling_tasks.yml

__Log:__
Log of the job and execution success:
[InfluxDB2-Downsample-Log.log](https://github.com/influxdata/influxdb/files/6022253/InfluxDB2-Downsample-Log.log)

Contributor guide

Open the contributing guide

Research direction

Start with the downsample-all-10m configuration in downsampling_tasks.yml and the attached execution log, then reproduce the regex filter behavior on InfluxDB 2.0.4 using the listed Telegraf and Docker setup. Compare matches when cpu is first versus later in the regex alternatives; done means the first partial match is handled consistently with the others.

Written by the indexing model from the issue text.

Assessment

Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.