influxdata / influxdata/influxdb
influxdb crashes when using the polyline.rdp() function with very large numbers.
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.
1. I use telegraf with the following input plugins
[[inputs.cpu]]
[[inputs.disk]]
[[inputs.diskio]]
2.-
The following query generates a database exception that causes the database to stop whenever there is data in the database that has a very large value (M, G). This causes the implementation of the algorithm to generate an exception that causes the database to crash. It has been tested on Linux and Windows with the same result. If this query is performed in a scheduled task, the database is corrupted and the influxdb service can no longer be restarted and the database has to be deleted to restart the service (tested on linux).
import "experimental/polyline"
from(bucket: "mde")
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
|> filter(fn: (r) => r["_measurement"] == "go_memstats_sys_bytes")
|> polyline.rdp()
with epsilon or retention argument the query works (not with small values)
__Expected behavior:__
Describe what you expected to happen.
El comportamiento esperado es el mismo que cuando se pone un parámetro epsilon y filtra los datos correctamente.
__Actual behavior:__
Describe What actually happened.
as described above
__Environment info:__
* System info: Run `uname -srm` and copy the output here
* InfluxDB version: Run `influxd version` and copy the output here
* Other relevant environment details: Container runtime, disk info, etc
In windows InfluxDB v2.6.1 (git: 9dcf880fe0) build_date: 2022-12-29T15:53:06Z
also same version in linux.
__Config:__
Copy any non-default config values here or attach the full config as a gist or file.
Standard configuration
__Logs:__
Include snippet of errors in log.
Log attached.
[errores.txt](https://github.com/influxdata/influxdb/files/10873834/errores.txt)
.....
....
.....
__Performance:__
Generate profiles with the following commands for bugs related to performance, locking, out of memory (OOM), etc.
```sh
# Commands should be run when the bug is actively happening.
# Note: This command will run for ~30 seconds.
curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=30s"
iostat -xd 1 30 > iostat.txt
# Attach the `profiles.tar.gz` and `iostat.txt` output files.
```
Contributor guide
Research direction
Start by reproducing the provided Flux query using experimental/polyline.rdp() with large values on InfluxDB v2.6.1, then inspect the attached errores.txt log and the polyline implementation. Done means the query no longer crashes or corrupts the database, while epsilon-based queries continue to work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100