influxdata / influxdata/influxdb
Points beyond retention policy dropped for infinite retention policy
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
__Steps to reproduce:__
I'm writing data to InfluxDB in bulk using python library. If I push data for some specific period it's giving me error saying that `400: {"error":"partial write: points beyond retention policy dropped=1"}`
I did little dig-up to find out why it's giving error for this specific period and found that some shards are overlapping. Can it be the reason?
I tried writing data one at a time and it's working for the same duration same data.
```
$ show shards
name: mydatabase
id database retention_policy shard_group start_time end_time expiry_time owners
-- -------- ---------------- ----------- ---------- -------- ----------- ------
447 mydatabase autogen 447 2018-10-22T00:00:00Z 2018-11-19T00:00:00Z 2018-11-19T00:00:00Z
321 mydatabase autogen 321 2018-11-12T00:00:00Z 2018-11-19T00:00:00Z 2018-11-19T00:00:00Z
449 mydatabase autogen 449 2018-11-19T00:00:00Z 2018-12-17T00:00:00Z 2018-12-17T00:00:00Z
67 mydatabase autogen 67 2018-12-24T00:00:00Z 2018-12-31T00:00:00Z 2018-12-31T00:00:00Z
3 mydatabase autogen 3 2018-12-31T00:00:00Z 2019-01-07T00:00:00Z 2019-01-07T00:00:00Z
453 mydatabase autogen 453 2018-12-17T00:00:00Z 2019-01-14T00:00:00Z 2019-01-14T00:00:00Z
4 mydatabase autogen 4 2019-01-07T00:00:00Z 2019-01-14T00:00:00Z 2019-01-14T00:00:00Z
6 mydatabase autogen 6 2019-01-14T00:00:00Z 2019-01-21T00:00:00Z 2019-01-21T00:00:00Z
8 mydatabase autogen 8 2019-01-21T00:00:00Z 2019-01-28T00:00:00Z 2019-01-28T00:00:00Z
10 mydatabase autogen 10 2019-01-28T00:00:00Z 2019-02-04T00:00:00Z 2019-02-04T00:00:00Z
```
Also check retention policy which is set to infinite.
```
> show retention policies
name duration shardGroupDuration replicaN default
---- -------- ------------------ -------- -------
autogen 0s 672h0m0s 1 true
```
I did changed shardGroupDuration multiple time. Can it cause issue?
__Expected behavior:__
Data should have been written in chunk as well.
__Actual behavior:__
Data is dropped by retention policy
__Environment info:__
* System info: Linux 4.4.0-1087-aws x86_64
* InfluxDB version: InfluxDB v1.7.6 (git: 1.7 01c8dd416270f424ab0c40f9291e269ac6921964)
Contributor guide
Assessment
This issue has not been assessed yet.