elastic / elastic/integrations

[field-mapping-conflicts] ti_strider drops failed indicator docs due to event.kind constant_keyword conflict

Open
#19,882 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
3d 4h
Merged PRs (30d)
209

Description

## Findings

### 1. `ti_strider` failed indicator documents are rejected after the failure handler changes `event.kind`

**Evidence:** `packages/ti_strider/data_stream/indicator/fields/base-fields.yml` declares `event.kind` as a `constant_keyword`:

```yaml
7. - name: event.kind
8. type: constant_keyword
9. description: High-level kind of the event (e.g. enrichment for threat indicators).
```

The same data stream's ingest pipeline first sets normal documents to `event.kind: enrichment` in `packages/ti_strider/data_stream/indicator/elasticsearch/ingest_pipeline/default.yml`:

```yaml
10. - set:
11. field: event.kind
12. value: enrichment
```

But its top-level `on_failure` handler sets the same field to `pipeline_error`:

```yaml
95. on_failure:
96. - set:
97. field: event.kind
98. value: pipeline_error
```

**What is wrong:** A `constant_keyword` field is indexed as one constant value for the index. After any processor failure in this pipeline, the failed document is rewritten with `event.kind: pipeline_error`, which conflicts with the normal `enrichment` value used by successful documents in the same data stream.

**Why it matters:** `on_failure` is intended to keep malformed source documents searchable with error context. Here, those failure-path documents can be rejected by Elasticsearch because the failure handler writes a value that the field mapping cannot hold, causing failed Strider indicator events to be dropped instead of indexed in a degraded state.

**Regression notes:** `git log --oneline -p -- packages/ti_strider/data_stream/indicator/fields/base-fields.yml` and the matching pipeline history show both the `constant_keyword` mapping and the `pipeline_error` failure handler were introduced together in `bd185c6c6e` (`ti_strider: add Strider Shield threat intelligence integration (#17025)`). This appears to be an original integration issue rather than a later downgrade.

**Suggested fix:** Keep `event.kind` as `constant_keyword` and stop changing it in the failure handler, storing failure state in `error.*`, `tags`, or `event.type` instead; or change this data stream's `event.kind` mapping to `keyword` if `pipeline_error` is an intentional indexed value.

## Suggested Actions

- [ ] Make the `ti_strider` indicator mapping and failure-path `event.kind` value consistent.
- [ ] Add a pipeline/indexing test with malformed input that exercises `on_failure` and confirms the document indexes with error context rather than being rejected.

Note: existing open issue #18001 tracks the same conflict pattern for other packages and Arista integer overflow cases, but it does not list `ti_strider`.

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Sweeper: Field Mapping Type Conflicts](https://github.com/elastic/integrations/actions/runs/28436860247)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Jul 7, 2026, 10:25 AM UTC

Contributor guide

Open the contributing guide

Research direction

Start with packages/ti_strider/data_stream/indicator/fields/base-fields.yml and packages/ti_strider/data_stream/indicator/elasticsearch/ingest_pipeline/default.yml to compare the event.kind mapping with the on_failure behavior. Run or add the suggested malformed-input pipeline/indexing test; done means failed documents index with error context without an event.kind mapping conflict.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch
Domain
search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.