elastic / elastic/integrations

[Github] Consider storing API Request events in a separated Data Stream

Open
#18,967 7 comments 0 reactions 1 assignee Claimed by @mohitjha-elastic View on GitHub
enhancement Integration:github Team:SDE-Crest Team:Security-Service Integrations
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
2d 17h
Merged PRs (30d)
225

Description

Hello,

When Github streams audit logs to a bucket, it send the audit records events and also the API request events related to your organizations.

Most security rules and dashboards will use the audit records events, but the volume of API Request events will be way higher than the audit records events, in my case more than 80% of the events are API Request events.

Since both are stored in the same data stream, the rollover is basically controlled by API Requests events which are the most noisy, but least used in rules and dashboards.

This impacts in the retention of the data.

Would be possible to split this and store the API Request events on a different data stream? Something like `github.api_requests` ? The filtering can be done in the end of the ingest pipeline as all API Request events have `event.action` = `api.request`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.