Graylog2 / Graylog2/graylog2-server

flex_parse_date() do not parse correctly ISO 8601 date with fraction of second

Open
#11,593 2 comments 1 reaction 0 assignees View on GitHub
bug processing triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

## Expected Behavior

I’m using pipelines to manipulate the date in incoming messages.

In my original message i got this field :
"timestamp_original":"2021-11-05T16:22:06.574735+01:00"

I’m parsing this field with flex_parse_date() and set the "timestamp" field with the result :
let new_date = flex_parse_date(to_string($message.timestamp_original));
set_field("timestamp", new_date);

I must got this in the field :
timestamp
2021-11-05T15:22:06.574735Z

## Current Behavior

But i got this in the result, which it wrong, the timezone is ignored :
timestamp
2021-11-05T16:22:06.000Z

For your information, theses date are parsed correctly :
2021-11-07T03:56:18+01:00
2021-11-07T02:56:12.848Z

## Possible Solution

## Steps to Reproduce (for bugs)

1. Add a pipeline like this :
```
rule "System - Replace the system field timestamp with timestamp_original"
when
true
then
let new_date = flex_parse_date(to_string($message.timestamp_original));
set_field("timestamp", new_date);
end
```
2. Put in the field "timestamp_original" an ISO 8601 date with fraction of second like this one : "2021-11-05T16:22:06.574873+01:00"
3. Run the pipeline on it
4. compare the "timestamp" field and the "timestamp_original"

## Context

I store in the message , the time when the message was created.
When the message is taken by Graylog, I replace content the "timestamp" field with the date of the "timestamp_original" field.

## Your Environment

* Graylog Version: 4.1.5 & 4.2.1
* Java Version: openjdk 11.0.13 2021-10-19 LTS
* Elasticsearch Version: 7.10.2
* MongoDB Version: 4.2.17
* Operating System: CentOS 8
* Browser version: Firefox 95

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.