elastic / elastic/integrations
[Github] Consider storing API Request events in a separated Data Stream
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 225
Description
Hello,
When Github streams audit logs to a bucket, it send the audit records events and also the API request events related to your organizations.
Most security rules and dashboards will use the audit records events, but the volume of API Request events will be way higher than the audit records events, in my case more than 80% of the events are API Request events.
Since both are stored in the same data stream, the rollover is basically controlled by API Requests events which are the most noisy, but least used in rules and dashboards.
This impacts in the retention of the data.
Would be possible to split this and store the API Request events on a different data stream? Something like `github.api_requests` ? The filtering can be done in the end of the ingest pipeline as all API Request events have `event.action` = `api.request`.
Contributor guide
Assessment
This issue has not been assessed yet.