[pipeline-error-handling] PANW PAN-OS pipelines silently suppress core parse failures

## Findings summary

I scanned ingest pipelines under `packages/*/data_stream/*/elasticsearch/ingest_pipeline/*.yml` and verified high-priority candidates.

- Total pipelines scanned: **1736**
- Missing `on_failure`: **209**
- Incomplete `on_failure` (missing `event.kind` and/or `error.message`): **213**

Open issue `#19357` already tracks major missing `event.kind` cases (`system`/`nginx`/`apache`), so this report focuses on a distinct high-impact gap.

## Findings

### 1. PANW child pipelines suppress primary parse errors, making failures invisible

**Evidence (locations):**

Primary parser in each child pipeline is a top-level `csv` processor with `ignore_failure: true`:

- `packages/panw/data_stream/panos/elasticsearch/ingest_pipeline/audit.yml:5-8`
- `packages/panw/data_stream/panos/elasticsearch/ingest_pipeline/authentication.yml:5-8`
- `packages/panw/data_stream/panos/elasticsearch/ingest_pipeline/correlated_event.yml:5-8`
- `packages/panw/data_stream/panos/elasticsearch/ingest_pipeline/decryption.yml:5-8`
- `packages/panw/data_stream/panos/elasticsearch/ingest_pipeline/gtp.yml:5-8`
- `packages/panw/data_stream/panos/elasticsearch/ingest_pipeline/ip_tag.yml:5-8`
- `packages/panw/data_stream/panos/elasticsearch/ingest_pipeline/sctp.yml:5-8`
- `packages/panw/data_stream/panos/elasticsearch/ingest_pipeline/system.yml:5-8`
- `packages/panw/data_stream/panos/elasticsearch/ingest_pipeline/tunnel_inspection.yml:5-8`

Example snippet (repeated pattern):

```yaml
- csv:
field: message
ignore_failure: true
```

Parent routing to these child pipelines:
- `packages/panw/data_stream/panos/elasticsearch/ingest_pipeline/default.yml:82-121`

Parent preserves original only when `ctx.error?.message != null`:
- `packages/panw/data_stream/panos/elasticsearch/ingest_pipeline/default.yml:2060-2065`

Each child does define `on_failure` with `event.kind: pipeline_error` and `error.message` (e.g. `audit.yml:42-49`), but that handler is bypassed when the first `csv` processor uses `ignore_failure: true`.

**What is wrong:**
For core parse failures in PAN-OS free-form log lines (malformed/truncated CSV, quoting drift, field-count drift), the first parser failure is swallowed, so no `error.message` is created and no `event.kind: pipeline_error` is emitted.

**Why it matters:**
These are high-volume network/security-device logs where parse failures are inevitable in production. Silent parser failures produce documents missing parsed fields with no pipeline-error signal, breaking operator visibility, triage, and detection reliability.

**Suggested fix:**
For the first `csv` processor in the affected PANW child pipelines, stop swallowing parser failure:

1. Replace `ignore_failure: true` with explicit `on_failure` that appends `error.message` and fails (or rethrows), so failure reaches pipeline error handling.
2. Keep/ensure `event.kind: pipeline_error` is set in failure path.
3. Ensure preserve-original behavior on failure is driven by explicit failure signaling instead of relying on silently suppressed processors.

## Suggested Actions

- [ ] Update PANW child pipelines listed above so core `csv` parse failures are not silently ignored.
- [ ] Add pipeline tests with malformed PAN-OS CSV inputs proving failure emits `event.kind: pipeline_error` and `error.message`.
- [ ] Confirm `preserve_original_event` behavior on PANW failure paths remains consistent with user configuration.

> [!NOTE]
>
> 🔒 Integrity filter blocked 18 items
>
> The following items were blocked because they don't meet the GitHub integrity level.
>
> - [#16563](https://github.com/elastic/integrations/issues/16563) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#14548](https://github.com/elastic/integrations/issues/14548) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#14269](https://github.com/elastic/integrations/issues/14269) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#16486](https://github.com/elastic/integrations/issues/16486) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#18342](https://github.com/elastic/integrations/issues/18342) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#2532](https://github.com/elastic/integrations/issues/2532) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#3102](https://github.com/elastic/integrations/issues/3102) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#12606](https://github.com/elastic/integrations/issues/12606) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#10975](https://github.com/elastic/integrations/issues/10975) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#15483](https://github.com/elastic/integrations/issues/15483) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#19039](https://github.com/elastic/integrations/issues/19039) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#18157](https://github.com/elastic/integrations/issues/18157) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#11830](https://github.com/elastic/integrations/issues/11830) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#17524](https://github.com/elastic/integrations/issues/17524) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#17059](https://github.com/elastic/integrations/issues/17059) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#17032](https://github.com/elastic/integrations/issues/17032) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - ... and 2 more items
>
> To allow these resources, lower `min-integrity` in your GitHub frontmatter:
>
> ```yaml
> tools:
> github:
> min-integrity: approved # merged | approved | unapproved | none
> ```
>
>

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Sweeper: Pipeline Error Handler Compliance](https://github.com/elastic/integrations/actions/runs/27269715010)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Jun 17, 2026, 10:37 AM UTC