Graylog2 / Graylog2/graylog2-server
`gl2_original_timestamp` causees data loss
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
Graylog now detects timestamps in the future and replaces them with a sane value, which is nice. But it stores the old value in `gl2_original_timestamp` and defines that field as a "date" in the index mapping. This causes index failures if the value contains an invalid date string.
Practical example: Some Kubernetes logging solution generated timestamps that look like this: `556925-01-21 05:58:28.000`. Probably because someone mixed up seconds and microseconds for a timestamp given to an iso date function.
## Expected Behavior
Graylog should detect all invalid `timestamp` values (not only dates in the future but also parsing errors) and replace broken timestamps with a known-good fallback value (e.g. `gl2_receive_timestamp`). It should NOT copy invalid dates into internal date fields that will later cause index errors anyway.
## Current Behavior
If Graylog receives a message with an invalid timestamp (e.g. `556925-01-21 05:58:28.000`) than it copies that string to `gl2_original_timestamp` which then causes index errors and data loss.
## Possible Solution
Define `gl2_original_timestamp` as a keyword or string field (which cannot be done by users via Field Type Profiles because the internal mapping type is hardcoded). Or have a new field called `gl2_invalid_timestamp` that is a string value and used if the original timestamp fails to parse. Or configure the mapping of `gl2_original_timestamp` as [ignore-malformed](https://docs.opensearch.org/docs/latest/field-types/mapping-parameters/ignore-malformed/). Throwing such events away is not a reasonable default action to take.
* Graylog Version: 6.2.3+b5538f9
* OpenSearch Version: 2.15.0
Contributor guide
Assessment
This issue has not been assessed yet.