elastic / elastic/integrations
[field-mapping-conflicts] ti_strider drops failed indicator docs due to event.kind constant_keyword conflict
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
## Findings
### 1. `ti_strider` failed indicator documents are rejected after the failure handler changes `event.kind`
**Evidence:** `packages/ti_strider/data_stream/indicator/fields/base-fields.yml` declares `event.kind` as a `constant_keyword`:
```yaml
7. - name: event.kind
8. type: constant_keyword
9. description: High-level kind of the event (e.g. enrichment for threat indicators).
```
The same data stream's ingest pipeline first sets normal documents to `event.kind: enrichment` in `packages/ti_strider/data_stream/indicator/elasticsearch/ingest_pipeline/default.yml`:
```yaml
10. - set:
11. field: event.kind
12. value: enrichment
```
But its top-level `on_failure` handler sets the same field to `pipeline_error`:
```yaml
95. on_failure:
96. - set:
97. field: event.kind
98. value: pipeline_error
```
**What is wrong:** A `constant_keyword` field is indexed as one constant value for the index. After any processor failure in this pipeline, the failed document is rewritten with `event.kind: pipeline_error`, which conflicts with the normal `enrichment` value used by successful documents in the same data stream.
**Why it matters:** `on_failure` is intended to keep malformed source documents searchable with error context. Here, those failure-path documents can be rejected by Elasticsearch because the failure handler writes a value that the field mapping cannot hold, causing failed Strider indicator events to be dropped instead of indexed in a degraded state.
**Regression notes:** `git log --oneline -p -- packages/ti_strider/data_stream/indicator/fields/base-fields.yml` and the matching pipeline history show both the `constant_keyword` mapping and the `pipeline_error` failure handler were introduced together in `bd185c6c6e` (`ti_strider: add Strider Shield threat intelligence integration (#17025)`). This appears to be an original integration issue rather than a later downgrade.
**Suggested fix:** Keep `event.kind` as `constant_keyword` and stop changing it in the failure handler, storing failure state in `error.*`, `tags`, or `event.type` instead; or change this data stream's `event.kind` mapping to `keyword` if `pipeline_error` is an intentional indexed value.
## Suggested Actions
- [ ] Make the `ti_strider` indicator mapping and failure-path `event.kind` value consistent.
- [ ] Add a pipeline/indexing test with malformed input that exercises `on_failure` and confirms the document indexes with error context rather than being rejected.
Note: existing open issue #18001 tracks the same conflict pattern for other packages and Arista integer overflow cases, but it does not list `ti_strider`.
---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Sweeper: Field Mapping Type Conflicts](https://github.com/elastic/integrations/actions/runs/28436860247)
Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Jul 7, 2026, 10:25 AM UTC
Contributor guide
Research direction
Start with packages/ti_strider/data_stream/indicator/fields/base-fields.yml and packages/ti_strider/data_stream/indicator/elasticsearch/ingest_pipeline/default.yml to compare the event.kind mapping with the on_failure behavior. Run or add the suggested malformed-input pipeline/indexing test; done means failed documents index with error context without an event.kind mapping conflict.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch
- Domain
- search
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100