elastic / elastic/integrations
[Github] New members data stream for entity support
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
## Summary
The GitHub integration currently collects security and operational data across six data
streams (`audit`, `issues`, `code_scanning`, `dependabot`, `secret_scanning`,
`security_advisories`), all of which are event or alert based. None of them collect
entity-state data; that is, a stable, current snapshot of who is a member of the
organization, what role they hold, which teams they belong to, and what their security
posture is.
This issue tracks the creation of a new `members` data stream that collects organization
member profiles and their associated attributes to support ECS `entity.*` field population
for GitHub user entities.
## Motivation
To support entity analytics workflows in Elastic (entity store, entity risk scoring, SIEM
identity graphs), the integration needs at least one data stream that represents entities
at rest rather than events over time. GitHub organization members are the primary identity
surface in GitHub and map directly to the ECS `user` entity type.
## Entity Type
The new data stream represents `user` entities, GitHub organization members (human
accounts and optionally bots). GitHub does not expose host or device entities via its API,
so `host.entity.*` fields are out of scope for this integration.
## Collection Strategy
The data stream will use a **GraphQL-first** approach, consistent with the existing
`dependabot` data stream which already uses `POST /graphql`. A single paginated GraphQL
query covers member identity, 2FA status, and team membership in one call. Supplementary
REST calls are used for membership role and optionally for owned repositories.
### Endpoints
| Call | Type | Purpose |
|------|------|---------|
| `POST https://api.github.com/graphql` / `organization.membersWithRole` | GraphQL | Primary collection: login, id, name, email, 2FA status, team memberships. Paginated via `pageInfo.endCursor`. |
| `GET /orgs/{org}/memberships/{username}` | REST | Per-member org role (`admin`, `member`, `billing_manager`) and effective permissions. |
| `GET /users/{username}/repos?type=owner` | REST | Repositories owned by the member. Opt-in O(n_members) calls. |
### Required OAuth Scopes (new)
| Scope | Reason |
|-------|--------|
| `read:org` | List members, read team membership, read org membership role |
| `read:user` | Read user profiles |
| `public_repo` or `repo` | Read owned repositories (only if the opt-in repos collection is enabled) |
Existing `read:audit_log` scope is unchanged.
## Manifest Variables (preliminary)
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `organization` | text | required | GitHub organization login name |
| `access_token` | password | required | PAT with `read:org` and `read:user` scopes |
| `interval` | text | `24h` | Polling interval |
| `collect_mfa_status` | bool | `false` | Enable 2FA status collection. Requires an org-owner token. |
| `collect_owned_repos` | bool | `false` | Enable per-member repository ownership collection. Adds O(n_members) REST calls per cycle. |
| `include_bots` | bool | `false` | Include bot accounts (type `Bot`) in collected members. |
Contributor guide
Assessment
This issue has not been assessed yet.