influxdata / influxdata/influxdb
In a database with a replication configured, In a written with points below and in the retention period none of the point is replicated.
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
__Steps to reproduce:__
1 Insert several points to the database, two of them beyond retention policy
influx write -b xxxxxxx -f /var/lib/influxdb2/test --org xxxx -t xxxxxx
TEST,xxxxx=1111,yyyy=22222 value=5.87452e-05 1685642375000000000
TEST,xxxxx=1111,yyyy=22222 value=4.87452e-05 1654091871000000000
TEST,xxxxx=1111,yyyy=22222 value=5.87452e-05 1654091861000000000
TEST,xxxxx=1111,yyyy=22222 value=4.87452e-05 1685627989000000000
Error: failed to write data: 422 Unprocessable Entity: failure writing points to database: partial write: points beyond retention policy dropped=2
2. Select the inserted point in the node where the replication is configured:
influx query 'from(bucket:\"xxxxxxx\") |> range(start:-1y) |> filter(fn: (r) => r._measurement == \"TEST\" )' --org xxxx -t xxxxxx
_start:time _stop:time _field:string _measurement:string xxxxx:string yyyy:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------- ---------------------- ------------------------------ ----------------------------
2022-06-05T02:34:06.487126506Z 2023-06-05T08:34:06.487126506Z value TEST 1111 22222 2023-06-01T13:59:49.000000000Z 0.0000487452
2022-06-05T02:34:06.487126506Z 2023-06-05T08:34:06.487126506Z value TEST 1111 22222 2023-06-01T17:59:35.000000000Z 0.0000587452
The two point inside the retention policy has been inserted in the bucket
3. Select the inserted point in the other node:
influx query 'from(bucket:\"xxxxxxx\") |> range(start:-1y) |> filter(fn: (r) => r._measurement == \"TEST\" )' --org xxxx -t xxxxxx
None of them has been replicated.
__Expected behavior:__
All the points written in the node where the replication is configured must be written to the other node.
__Actual behavior:__
If in a written with points below and in the retention period, the point correctly write in the database are not replicated
__Environment info:__
Running in a docker container, tested in
* System info: Linux 3.10.0-1160.el7.x86_64 x86_64
* InfluxDB version: Influxdb 2.6.1 and 2.7
* Other relevant environment details: Docker container runtime
__Config:__
#Disable the /debug/pprof HTTP endpoint.
#This endpoint provides runtime profiling data and can be helpful when debugging.
pprof-disabled: true
#Disable the HTTP /metrics endpoint which exposes internal InfluxDB metrics.
metrics-disabled: true
#Disable the InfluxDB user interface (UI). The UI is enabled by default.
ui-disabled: true
#Path to the BoltDB database. InfluxDB uses BoltDB to store data including organization and user information,
#UI data, REST resources, and other key value data.
bolt-path: /var/lib/influxdb2/influxd.bolt
#Path to persistent storage engine files where InfluxDB stores all Time-Structure Merge Tree (TSM) data on disk.
engine-path: /var/lib/influxdb2/engine
#Path to the SQLite database file. The SQLite database is used to store metadata for notebooks and annotations.
sqlite-path: /var/lib/influxdb2/influxd.sqlite
Contributor guide
Research direction
Reproduce the behavior with the supplied influx write and influx query commands using points both inside and beyond the retention period. Compare the source and other node, then trace the replication path for partial writes; done means points retained on the source are also present on the other node.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, linux
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100