elastic / elastic/integrations

[sentinel_one] Consolidate the integration onto a single CEL input

Open
#20,286 3 comments 0 reactions 0 assignees View on GitHub
enhancement Integration:sentinel_one Team:Security-Service Integrations
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
2d 17h
Merged PRs (30d)
225

Description

## The problem (user's view)

Setting up SentinelOne is more confusing than it should be. The integration is split across two Elastic Agent inputs, HTTPJSON and CEL, and that split leaks straight into the setup experience. A user has to configure the same SentinelOne console URL and API token more than once, and pick which data streams they want in two separate places depending on which input each one happens to use. Nothing about SentinelOne requires this. It's just history: streams were added at different times, and the older ones landed on HTTPJSON before CEL became our default.

So people set up what looks like one integration but feels like two, and they have to understand two different collection engines to reason about it. That's a bad first impression for a security source this popular.

## Where we already are

Good news: most of the work is done. Of the 9 data streams, 4 are already on CEL, and the `threat` stream was moved over in v2.9.0 (it still shows as HTTPJSON in Fleet but runs CEL underneath). Only four streams are still pure HTTPJSON: `activity`, `agent`, `alert`, and `group`.

Those four are the simplest ones we have. Each is a single API call with basic pagination and no enrichment, so they're the easiest possible things to migrate. We're finishing a job, not starting one.

## Proposal

Move `activity`, `agent`, `alert`, and `group` onto CEL, using the same pattern that worked for the `threat` stream. When it's done, the integration collects everything through one input. The user enters their connection details once, picks their streams in one place, and never has to think about input types at all.

## Why it's worth doing

- One clean setup instead of a split one. This is the whole point.
- One engine to explain in the docs and in support, instead of two.
- The CEL-only improvements we've already shipped (state persistence, dedup) start covering every stream instead of just some.

## What to watch out for

The real risk is the upgrade, not the build. Switching a stream's input type can reset its cursor, which could cause a gap or duplicate events right at the cutover. We need a plan for that and tests to prove it before release, and the changelog has to spell out what existing users should expect.

## Rough breakdown

1. Migrate `activity`, `agent`, `alert`, `group` to CEL (one issue each).
2. Prove the upgrade is clean: no gaps, no duplicates at cutover.
3. Update the docs and changelog to explain the single-input setup.

## Open question
Is this a breaking change or can we seamlessly switch inputs under the hood?
Once moved to CEL, can we hide the old httpjson based datastream and present users with just the CEL based datastream?

Contributor guide

Open the contributing guide

Research direction

Start by tracing the existing CEL implementation for the threat stream, then compare it with the activity, agent, alert, and group streams currently using HTTPJSON. Done means all four run through one CEL input, upgrade behavior is tested for cursor gaps or duplicate events, and the documentation and changelog explain the transition.

Written by the indexing model from the issue text.

Assessment

Domain
devops, security
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.