Graylog2 / Graylog2/graylog2-server
Extract field from json_extracted field not working
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
I have 2 extractors.
1. extract json
```
{
"title": "json_extract",
"extractor_type": "json",
"converters": [],
"order": 0,
"cursor_strategy": "copy",
"source_field": "message",
"target_field": "",
"extractor_config": {
"flatten": false,
"list_separator": ", ",
"kv_separator": "=",
"key_prefix": "xxx.",
"key_separator": ".",
"replace_key_whitespace": false,
"key_whitespace_replacement": "_"
},
"condition_type": "none",
"condition_value": ""
},
```
send (lower) in order:
```
{
"title": "split xxx_datetime_date as store as timestamp_xxx",
"extractor_type": "split_and_index",
"converters": [
{
"type": "flexdate",
"config": {
"time_zone": "Poland"
}
}
],
"order": 2,
"cursor_strategy": "copy",
"source_field": "xxx_datetime_date",
"target_field": "timestamp_xxx",
"extractor_config": {
"index": 1,
"split_by": "."
},
"condition_type": "string",
"condition_value": "."
}
```
part of the json extract:
{...
"datetime":{"date":"2017-10-10 09:39:01.607127","timezone_type":3,"timezone":"Europe/Warsaw"}
...}
The fields are extracted to:
xxx_datetime_date: 2017-10-10 09:39:01.607127
xxx_timezone_type: 3
xxx_timezone: Europe/Warsaw
## Expected Behavior
Json Extractor works fine. But When I try to extract again field xxx_datetime_date to timestamp_xxx
or just copy the field to anoter filed. It's not working.
## Current Behavior
Second extractor on extracted json field (xxx_datetime_date) does not work.
No additional field are added to message.
Pipelines also not working.
```
rule "Appserver Parsing - Timestamp"
when
has_field("xxx_datetime_date")
then
let new_timestamp = parse_date(to_string($message.xxx_datetime_date), "yyyy-MM-dd HH:mm:ss.SSS");
set_field("xxx_pipline_timestamp", new_timestamp);
// If the timestamp is correct, rename the field
end
```
## Context
I cannot set timestamp from messages taht come as full json.
* Graylog Version: 2.3.0
* Elasticsearch Version: /usr/bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/elasticsearch-2.4.4.jar:/usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -Des.pidfile=/var/run/elasticsearch/elasticsearch.pid -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.conf=/etc/elasticsearch
* MongoDB Version: v2.4.10
* Operating System: Ubuntu 16.10
* Browser version: Firefox 57 b5
Contributor guide
Assessment
This issue has not been assessed yet.