elastic / elastic/integrations
[pipeline-error-handling] HAProxy log pipeline fails to classify ingest failures as pipeline_error
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 225
Description
## Findings
### 1. HAProxy `log` ingest pipeline has incomplete top-level `on_failure`
**Location**
- `packages/haproxy/data_stream/log/elasticsearch/ingest_pipeline/default.yml:211-214`
- Related context: `packages/haproxy/data_stream/log/elasticsearch/ingest_pipeline/default.yml:13-40` (grok-heavy parsing), `:158-160` (`event.kind` is set to `event` in normal flow)
**Evidence**
```yaml
on_failure:
- set:
field: error.message
value: '\{\{ _ingest.on_failure_message }}'
```
The failure handler sets `error.message` but does not set `event.kind: pipeline_error`.
**What is wrong**
When parsing fails, the pipeline does not classify the document as a pipeline failure event. In this pipeline, normal processing sets `event.kind: event` (`:158-160`), so failure-path documents are not explicitly marked as `pipeline_error`.
**Why it matters**
`haproxy/log` is a free-form log source with multiple grok patterns (`:13-40`), where parse failures are expected in real deployments. Without `event.kind: pipeline_error`, failures are much harder to discover and triage at scale, even when `error.message` exists.
**Suggested fix**
Add `event.kind: pipeline_error` to the top-level `on_failure` block while keeping the existing `error.message` assignment, for example:
```yaml
on_failure:
- set:
field: event.kind
value: pipeline_error
- set:
field: error.message
value: '\{\{ _ingest.on_failure_message }}'
```
## Scan summary
- Pipelines scanned: **1667**
- Missing `on_failure`: **207**
- Incomplete `on_failure` (missing `event.kind: pipeline_error` and/or `error.message`): **212**
## Notes on de-duplication
- Existing open tracker `#18026` already covers `system/syslog`, `vsphere/log`, and `sonicwall_firewall` preserve behavior, so those were intentionally excluded from this report.
## Suggested actions
- [ ] Add `event.kind: pipeline_error` to `packages/haproxy/data_stream/log/elasticsearch/ingest_pipeline/default.yml` top-level `on_failure`.
- [ ] Add a regression test/assertion to ensure HAProxy ingest failures are emitted with `event.kind: pipeline_error`.
---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Sweeper: Pipeline Error Handler Compliance](https://github.com/elastic/integrations/actions/runs/24770882634)
Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Apr 29, 2026, 9:48 AM UTC
Contributor guide
Assessment
This issue has not been assessed yet.