elastic / elastic/integrations

[Microsoft Defender XDR]: Microsoft Defender XDR fails with HTTP 500 when using expand=alerts on large tenants

Open
#18,277 4 comments 0 reactions 1 assignee Claimed by @kcreddy View on GitHub
Integration:m365_defender needs:triage Team:Security-Service Integrations
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
3d 4h
Merged PRs (30d)
209

Description

### Integration Name

Microsoft Defender XDR

### Dataset Name

incidents

### Integration Version

5.12.3

### Agent Version

9.3

### OS Version and Architecture

All

### User Goal

The M365 Defender integration's incident data stream hardcodes $expand=alerts in its Microsoft Graph Security API query. On a large tenant (~45k users), this triggers persistent HTTP 500 InternalServerError responses from Microsoft when incidents have many linked alerts. The HTTP JSON input exhausts its retry budget (6 attempts), enters Degraded state, and the cursor never advances — creating a permanent ingestion stall.

Solutions:

1. Migrate incident data stream from HTTP JSON to CEL. The HTTP JSON template is doing straightforward work (cursor-based polling, pagination, nested split) that CEL can express directly. Migration gains:

Error resilience — no permanent stall on persistent API errors.
Two-step collection becomes natural: fetch incidents without $expand=alerts, then fetch alerts per-incident via GET /v1.0/security/incidents/{id}?$expand=alerts. This avoids the $filter + $expand trigger entirely.
Aligns with the vulnerability data stream, which already uses CEL.

2. Since it is already ingesting alerts using a separate data streams, we could use lookup joins to create relationships between the data.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.