influxdata / influxdata/influxdb

In a database with a replication configured, In a written with points below and in the retention period none of the point is replicated.

Open
#24,263 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

__Steps to reproduce:__
1 Insert several points to the database, two of them beyond retention policy
influx write -b xxxxxxx -f /var/lib/influxdb2/test --org xxxx -t xxxxxx
TEST,xxxxx=1111,yyyy=22222 value=5.87452e-05 1685642375000000000
TEST,xxxxx=1111,yyyy=22222 value=4.87452e-05 1654091871000000000
TEST,xxxxx=1111,yyyy=22222 value=5.87452e-05 1654091861000000000
TEST,xxxxx=1111,yyyy=22222 value=4.87452e-05 1685627989000000000

Error: failed to write data: 422 Unprocessable Entity: failure writing points to database: partial write: points beyond retention policy dropped=2
2. Select the inserted point in the node where the replication is configured:
influx query 'from(bucket:\"xxxxxxx\") |> range(start:-1y) |> filter(fn: (r) => r._measurement == \"TEST\" )' --org xxxx -t xxxxxx
_start:time _stop:time _field:string _measurement:string xxxxx:string yyyy:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------- ---------------------- ------------------------------ ----------------------------
2022-06-05T02:34:06.487126506Z 2023-06-05T08:34:06.487126506Z value TEST 1111 22222 2023-06-01T13:59:49.000000000Z 0.0000487452
2022-06-05T02:34:06.487126506Z 2023-06-05T08:34:06.487126506Z value TEST 1111 22222 2023-06-01T17:59:35.000000000Z 0.0000587452

The two point inside the retention policy has been inserted in the bucket

3. Select the inserted point in the other node:
influx query 'from(bucket:\"xxxxxxx\") |> range(start:-1y) |> filter(fn: (r) => r._measurement == \"TEST\" )' --org xxxx -t xxxxxx

None of them has been replicated.

__Expected behavior:__
All the points written in the node where the replication is configured must be written to the other node.

__Actual behavior:__
If in a written with points below and in the retention period, the point correctly write in the database are not replicated
__Environment info:__
Running in a docker container, tested in

* System info: Linux 3.10.0-1160.el7.x86_64 x86_64
* InfluxDB version: Influxdb 2.6.1 and 2.7
* Other relevant environment details: Docker container runtime

__Config:__
#Disable the /debug/pprof HTTP endpoint.
#This endpoint provides runtime profiling data and can be helpful when debugging.
pprof-disabled: true
#Disable the HTTP /metrics endpoint which exposes internal InfluxDB metrics.
metrics-disabled: true
#Disable the InfluxDB user interface (UI). The UI is enabled by default.
ui-disabled: true
#Path to the BoltDB database. InfluxDB uses BoltDB to store data including organization and user information,
#UI data, REST resources, and other key value data.
bolt-path: /var/lib/influxdb2/influxd.bolt
#Path to persistent storage engine files where InfluxDB stores all Time-Structure Merge Tree (TSM) data on disk.
engine-path: /var/lib/influxdb2/engine
#Path to the SQLite database file. The SQLite database is used to store metadata for notebooks and annotations.
sqlite-path: /var/lib/influxdb2/influxd.sqlite

Contributor guide

Open the contributing guide

Research direction

Reproduce the behavior with the supplied influx write and influx query commands using points both inside and beyond the retention period. Compare the source and other node, then trace the replication path for partial writes; done means points retained on the source are also present on the other node.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, linux
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.