elastic / elastic/logstash

Datatype difference on geoip.location after updating to 5.4.2

Open
#7,569 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
19h 14m
Merged PRs (30d)
63

Description

Hi, I have a ES database, populated via logstash, which stores web server logs, mail server logs, platform logs and adds them additional information such as GeoIP coordinates when a IP address is found.

While doing a revision over it and trying to perform a reindex on one of the indexes, elasticsearch failed with the following message:

```
"failures" : [
{
"index" : "reindexed",
"type" : "apache",
"id" : "AVzXZrIdayxwu2TPqhb2",
"cause" : {
"type" : "mapper_parsing_exception",
"reason" : "failed to parse [geoip.location]",
"caused_by" : {
"type" : "json_parse_exception",
"reason" : "Current token (START_OBJECT) not numeric, can not use numeric value accessors\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@72502245; line: 1, column: 531]"
}
},
"status" : 400
},
```

while the indexes from previous days were copied without error.

It seems that after updating logstash (the whole stack, actually) to 5.4.2, all the entries are created with the `geoip.location` field as
```
geoip.location: { "lon": DECIMAL, "lat": DECIMAL }
```
for example
```
geoip.location: { "lon": -60.7351, "lat": -31.7351 }
```
while the previous entries (logstash 5.4.1 and previous) were added as
```
geoip.location: DECIMAL, DECIMAL
```
for example
```
geoip.location: -60.7351, -31.7351
```
As such, I have indexes with slightly different mappings. Unfortunately, I am still new to this platform and not able to define if this difference on the behavior of 5.4.1 and 5.4.2 is due to:
- a bug on logstash (or logstash-plugins);
- a change done on purpose, as the Release Notes for that version show changes on GeoIP filters;
- a misconfiguration on my behalf.

If such, I would gladly follow the advice of any experienced user who could provide a way to convert the fields and/or resolve the issue.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.