elastic / elastic/ml-cpp

[ML] lat_long anomaly not detected anymore

Open
#2,162 0 comments 0 reactions 0 assignees View on GitHub
:ml
Dominant language
C++
Stars
157
Forks
67
Avg merge
12h 48m
Merged PRs (30d)
16

Description

#### Summary

We used to have a job configuration that produced an anomaly and now it doesn't do that anymore, so we'd like to assess whether or not this is an expected change / a change we want to tolerate.

### Steps to reproduce

1. Install the Kibana ecommerce sample data
2. Create and run the anomaly detection lookback job (synchronize Kibana saved objects if needed)

Config

```
PUT _ml/anomaly_detectors/ecommerce-geo
{
"analysis_config" : {
"bucket_span":"15m",
"detectors": [
{
"detector_description": "Unusual coordinates by user",
"function": "lat_long",
"field_name": "geoip.location",
"by_field_name": "user"
}
],
"influencers": [
"geoip.country_iso_code",
"day_of_week",
"category.keyword",
"user"
]
},
"data_description" : {
"time_field": "order_date"
},
"datafeed_config":{
"datafeed_id": "datafeed-ecommerce-geo",
"indices": ["kibana_sample_data_ecommerce"],
"query": {
"bool": {
"must": [
{
"match_all": {}
}
]
}
}
}
}
```

3. View the job results

### Additional information

- This job config was used to create screenshots for the docs, e.g. the first one in [this section](https://www.elastic.co/guide/en/machine-learning/master/geographic-anomalies.html#geographic-anomalies-results):
![image](https://user-images.githubusercontent.com/1945390/146507541-91e7deb7-5278-4cfc-b49b-34312d2aabde.png)
- When looking at the data for `user: jackson`, we can see 114 documents with 113 of them having the same `geoip.location` somewhere near Los Angeles and one of them having a `geoip.location` near New York
![image](https://user-images.githubusercontent.com/1945390/146509077-a74b6a84-894c-43bc-bdc3-4c98e874cc96.png)
- We can also see this by running a high precision geohash grid aggregation:
```
GET kibana_sample_data_ecommerce/_search
{
"query": {
"simple_query_string": {
"query": "jackson",
"fields": ["user"]
}
},
"aggs": {
"locations": {
"geohash_grid": {"field": "geoip.location", "precision": 12}
}
},
"size": 0
}
```
which gives us
```
[...]
"locations" : {
"buckets" : [
{
"key" : "9q5cyr9qukez",
"doc_count" : 113
},
{
"key" : "dr5rs14yejbs",
"doc_count" : 1
}
]
}
```
- So from this data, the originally detected anomaly seems correct.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.