elastic / elastic/package-spec
[Change Proposal] Support look back time for TSDB indexes
- Dominant language
- Go
- Stars
- 20
- Forks
- 93
- Avg merge
- 15h 10m
- Merged PRs (30d)
- 12
Description
Time series data streams have a writable window of 2 hours in the past. This means that documents whose `@timestamp` is older than these 2 hours are rejected.
There are integrations use cases where data is collected in longer periods, for example metrics that are published only daily. In these cases the 2 hours window is too small.
Elasticsearch supports [`index.look_back_time`](https://www.elastic.co/docs/reference/elasticsearch/index-settings/time-series) to change this 2 hours default.
Allow to use `index.look_back_time` in the index settings. This should be only allowed when time series is enabled.
For completitude, allow to use also `index.look_ahead_time`, that increases the window in the future.
As a workaround users can already override this value using `@custom` index templates, but it should work out of the box for integrations that are known to collect data older than 2 hours.
Feature requested in https://github.com/elastic/integrations/pull/19339#issuecomment-4613358008.
Contributor guide
Assessment
This issue has not been assessed yet.