influxdata / influxdata/influxdb
InfluxDB2 Task - First Regex Match Being Missed (Tasks)
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.
1. Apply/Utilise Default Configuration for Telegraf Windows
2. Import Task to Downsample Data (Downsample All) as per https://github.com/influxdata/community-templates/tree/master/downsampling
3. Execute Task i.e. "downsample-all-10m" against specified Source (From Bucket) and Destination (To Bucket)
__Expected behavior:__
All fields where there is a partial match using the supplied regex should be caught and downsampled.
__Actual behavior:__
The first field in the regex partial match is not executed by the task
This can be demonstrated by either:
1. Rearranging the partial regex to demonstrate that the match occurs successfully as long as it is not first. However, whatever is now first will be missed further proving the issue.
2. Entering a random unique string to effectively be caught first and allow all others to be matched works but is non-desirable
i.e.:
all_data = from(bucket: fromBucket)
|> range(start: -task.every)
|> filter(fn: (r) =>
(r._measurement =~ /^cpu|disk|diskio|mem|processes|swap|system|internal_.+|net|netstat|procstat|procstat_lookup|smart_.+$/))
Using the supplied downsample above, any measurement which contains "cpu" is missed. However, using either of the above options of rearranging the regex so that "cpu" isn't first or entering a dummy value at the start will enable "cpu" to match successfully.
It appears for whatever reason, InfluxDB2 is missing the first statement in the regex where measurement is assigned and then executed. As long as it is not the first measurement, it is downsampled correctly.
__Environment info:__
* System info: docker exec -ti influxdb uname -srm
Linux 5.4.0-64-generic x86_64
* InfluxDB version: docker exec -ti influxdb influxd
2021-02-22T11:55:29.944943Z info Welcome to InfluxDB {"log_id": "0SUPMJ60000", "version": "2.0.4", "commit": "4e7a59bb9a", "build_date": "2021-02-08T17:47:02Z"}
2021-02-22T11:55:30.901368Z error Failed opening bolt {"log_id": "0SUPMJ60000", "error": "unable to open boltdb: timeout"}
Error: unable to open boltdb: timeout
_Not sure why the error related to boltdb occurs._
* Other relevant environment details:
InfluxDB Docker Container - quay.io/influxdb/influxdb
__Config:__
Telegraf Config - https://github.com/influxdata/telegraf/blob/master/etc/telegraf.conf
Downsample Config (I've utilised downsample-all-10m) - https://github.com/influxdata/community-templates/blob/master/downsampling/all_inputs/downsampling_tasks.yml
__Log:__
Log of the job and execution success:
[InfluxDB2-Downsample-Log.log](https://github.com/influxdata/influxdb/files/6022253/InfluxDB2-Downsample-Log.log)
Contributor guide
Research direction
Start with the downsample-all-10m configuration in downsampling_tasks.yml and the attached execution log, then reproduce the regex filter behavior on InfluxDB 2.0.4 using the listed Telegraf and Docker setup. Compare matches when cpu is first versus later in the regex alternatives; done means the first partial match is handled consistently with the others.
Written by the indexing model from the issue text.
Assessment
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100