Drop events without required fields in Elasticsearch
- Dominant language
- Go
- Stars
- 1.3k
- Forks
- 543
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 109
Description
The Intake API in APM Server already requires certain fields to be set, and will drop events without them. This makes it possible for APM UI to have a stricter data model where some fields can be assumed to exist.
Despite that we still receive sdh’s where the customer is seeing errors caused by invalid documents (aka documents without required fields).
In a recent example a customer was removing `agent.name` in a logstash pipeline: https://github.com/elastic/sdh-apm/issues/928#issuecomment-1549106087
This is quite time consuming to debug, and it would be great if we could prevent this from happening. Would it be possibly to require the same fields in ES via a pipeline that is auto-generated by APM Server?
Contributor guide
Assessment
This issue has not been assessed yet.