elastic / elastic/integrations
[ti_otx] Consolidate the integration onto a single CEL input
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
# Consolidate the AlienVault OTX integration onto a single CEL input
**Package:** `ti_otx` · **Type:** Enhancement · **Team:** Security Service Integrations
## The problem (user's view)
The integration is split across two Elastic Agent inputs. `pulses_subscribed` runs on CEL, but `threat` is still on HTTPJSON, and each input carries its own copy of the connection config — so a user pointing at OTX enters the same API key and settings twice and picks streams in two places. Nothing about OTX requires this; it's just that `threat` was never migrated.
## Where we are
Two data streams (v1.32.1): `pulses_subscribed` on CEL, `threat` on HTTPJSON. Finishing the job here means migrating one stream.
## What I'm proposing
Migrate `threat` to CEL so everything runs through one input and the connection is configured once.
## Why it's worth doing
- Credentials and setup entered once instead of twice.
- One engine to explain in docs and support.
- CEL-only improvements (state persistence, dedup, error handling) cover both streams.
## What to watch out for
The upgrade is the risky part. Switching `threat` from HTTPJSON to CEL can reset its cursor and cause a gap or duplicate events at cutover, so we need a cursor-continuity plan, tests that prove no gaps/duplicates, and a changelog note for existing users.
## Rough breakdown
1. Migrate `threat` to CEL + tests.
2. Prove the upgrade is clean: no gaps, no duplicates at cutover.
3. Update docs and changelog.
Part of a wider push to put Security Service Integrations packages on a single CEL input. Related: #20286 (SentinelOne), #14813 (Google Workspace), #20288 (Cortex XDR).
Contributor guide
Research direction
Start by locating the ti_otx definitions for the pulses_subscribed CEL stream and the threat HTTPJSON stream, then compare their connection and cursor behavior. Verify the migration with tests covering cursor continuity, no gaps, and no duplicate events at cutover; finish with documentation and a changelog note for existing users.
Written by the indexing model from the issue text.
Assessment
- Domain
- backend, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100