elastic / elastic/integrations
[sentinel_one] Consolidate the integration onto a single CEL input
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 225
Description
## The problem (user's view)
Setting up SentinelOne is more confusing than it should be. The integration is split across two Elastic Agent inputs, HTTPJSON and CEL, and that split leaks straight into the setup experience. A user has to configure the same SentinelOne console URL and API token more than once, and pick which data streams they want in two separate places depending on which input each one happens to use. Nothing about SentinelOne requires this. It's just history: streams were added at different times, and the older ones landed on HTTPJSON before CEL became our default.
So people set up what looks like one integration but feels like two, and they have to understand two different collection engines to reason about it. That's a bad first impression for a security source this popular.
## Where we already are
Good news: most of the work is done. Of the 9 data streams, 4 are already on CEL, and the `threat` stream was moved over in v2.9.0 (it still shows as HTTPJSON in Fleet but runs CEL underneath). Only four streams are still pure HTTPJSON: `activity`, `agent`, `alert`, and `group`.
Those four are the simplest ones we have. Each is a single API call with basic pagination and no enrichment, so they're the easiest possible things to migrate. We're finishing a job, not starting one.
## Proposal
Move `activity`, `agent`, `alert`, and `group` onto CEL, using the same pattern that worked for the `threat` stream. When it's done, the integration collects everything through one input. The user enters their connection details once, picks their streams in one place, and never has to think about input types at all.
## Why it's worth doing
- One clean setup instead of a split one. This is the whole point.
- One engine to explain in the docs and in support, instead of two.
- The CEL-only improvements we've already shipped (state persistence, dedup) start covering every stream instead of just some.
## What to watch out for
The real risk is the upgrade, not the build. Switching a stream's input type can reset its cursor, which could cause a gap or duplicate events right at the cutover. We need a plan for that and tests to prove it before release, and the changelog has to spell out what existing users should expect.
## Rough breakdown
1. Migrate `activity`, `agent`, `alert`, `group` to CEL (one issue each).
2. Prove the upgrade is clean: no gaps, no duplicates at cutover.
3. Update the docs and changelog to explain the single-input setup.
## Open question
Is this a breaking change or can we seamlessly switch inputs under the hood?
Once moved to CEL, can we hide the old httpjson based datastream and present users with just the CEL based datastream?
Contributor guide
Research direction
Start by tracing the existing CEL implementation for the threat stream, then compare it with the activity, agent, alert, and group streams currently using HTTPJSON. Done means all four run through one CEL input, upgrade behavior is tested for cursor gaps or duplicate events, and the documentation and changelog explain the transition.
Written by the indexing model from the issue text.
Assessment
- Domain
- devops, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100