elastic / elastic/integrations

[ti_otx] Consolidate the integration onto a single CEL input

Open
#20,289 3 comments 0 reactions 0 assignees View on GitHub
enhancement Integration:ti_otx Team:Security-Service Integrations
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
3d 4h
Merged PRs (30d)
209

Description

# Consolidate the AlienVault OTX integration onto a single CEL input

**Package:** `ti_otx` · **Type:** Enhancement · **Team:** Security Service Integrations

## The problem (user's view)
The integration is split across two Elastic Agent inputs. `pulses_subscribed` runs on CEL, but `threat` is still on HTTPJSON, and each input carries its own copy of the connection config — so a user pointing at OTX enters the same API key and settings twice and picks streams in two places. Nothing about OTX requires this; it's just that `threat` was never migrated.

## Where we are
Two data streams (v1.32.1): `pulses_subscribed` on CEL, `threat` on HTTPJSON. Finishing the job here means migrating one stream.

## What I'm proposing
Migrate `threat` to CEL so everything runs through one input and the connection is configured once.

## Why it's worth doing
- Credentials and setup entered once instead of twice.
- One engine to explain in docs and support.
- CEL-only improvements (state persistence, dedup, error handling) cover both streams.

## What to watch out for
The upgrade is the risky part. Switching `threat` from HTTPJSON to CEL can reset its cursor and cause a gap or duplicate events at cutover, so we need a cursor-continuity plan, tests that prove no gaps/duplicates, and a changelog note for existing users.

## Rough breakdown
1. Migrate `threat` to CEL + tests.
2. Prove the upgrade is clean: no gaps, no duplicates at cutover.
3. Update docs and changelog.

Part of a wider push to put Security Service Integrations packages on a single CEL input. Related: #20286 (SentinelOne), #14813 (Google Workspace), #20288 (Cortex XDR).

Contributor guide

Open the contributing guide

Research direction

Start by locating the ti_otx definitions for the pulses_subscribed CEL stream and the threat HTTPJSON stream, then compare their connection and cursor behavior. Verify the migration with tests covering cursor continuity, no gaps, and no duplicate events at cutover; finish with documentation and a changelog note for existing users.

Written by the indexing model from the issue text.

Assessment

Domain
backend, security
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.