influxdata / influxdata/influxdb

influxdb crashes when using the polyline.rdp() function with very large numbers.

Open
#24,121 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.

1. I use telegraf with the following input plugins
[[inputs.cpu]]
[[inputs.disk]]
[[inputs.diskio]]

2.-
The following query generates a database exception that causes the database to stop whenever there is data in the database that has a very large value (M, G). This causes the implementation of the algorithm to generate an exception that causes the database to crash. It has been tested on Linux and Windows with the same result. If this query is performed in a scheduled task, the database is corrupted and the influxdb service can no longer be restarted and the database has to be deleted to restart the service (tested on linux).

import "experimental/polyline"
from(bucket: "mde")
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
|> filter(fn: (r) => r["_measurement"] == "go_memstats_sys_bytes")
|> polyline.rdp()

with epsilon or retention argument the query works (not with small values)

__Expected behavior:__
Describe what you expected to happen.

El comportamiento esperado es el mismo que cuando se pone un parámetro epsilon y filtra los datos correctamente.
__Actual behavior:__
Describe What actually happened.

as described above

__Environment info:__

* System info: Run `uname -srm` and copy the output here
* InfluxDB version: Run `influxd version` and copy the output here
* Other relevant environment details: Container runtime, disk info, etc

In windows InfluxDB v2.6.1 (git: 9dcf880fe0) build_date: 2022-12-29T15:53:06Z
also same version in linux.

__Config:__
Copy any non-default config values here or attach the full config as a gist or file.

Standard configuration

__Logs:__
Include snippet of errors in log.

Log attached.

[errores.txt](https://github.com/influxdata/influxdb/files/10873834/errores.txt)

.....
....
.....

__Performance:__
Generate profiles with the following commands for bugs related to performance, locking, out of memory (OOM), etc.

```sh
# Commands should be run when the bug is actively happening.
# Note: This command will run for ~30 seconds.
curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=30s"
iostat -xd 1 30 > iostat.txt
# Attach the `profiles.tar.gz` and `iostat.txt` output files.
```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the provided Flux query using experimental/polyline.rdp() with large values on InfluxDB v2.6.1, then inspect the attached errores.txt log and the polyline implementation. Done means the query no longer crashes or corrupts the database, while epsilon-based queries continue to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.