elastic / elastic/integrations
[docs-patrol] Document AWS VPC Flow unsupported-format pipeline_error behavior
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
Recent code changes in the repository have introduced documentation drift. The following change needs a corresponding documentation update.
## Changes Requiring Documentation Updates
### 1. AWS VPC Flow now emits pipeline errors for unsupported plain-text formats
**Commit(s):**
- [`ddd75707f42fb6fbea02af51f4fec692f842b5a7`](https://github.com/elastic/integrations/commit/ddd75707f42fb6fbea02af51f4fec692f842b5a7)
**What changed:**
- The VPC Flow ingest pipeline added a `fail` processor for records whose token count does not match supported layouts and where parsing did not populate `aws.vpcflow.srcaddr`.
- This changed behavior from silently indexing unsupported formats unparsed to surfacing them as pipeline errors.
**Evidence:**
- `packages/aws/data_stream/vpcflow/elasticsearch/ingest_pipeline/default.yml:72-77` adds:
- `fail` on `ctx?._temp_?.message_token_count != null && ctx?.aws?.vpcflow?.srcaddr == null`
- message: `Unsupported VPC Flow Log format... Use a custom ingest pipeline to parse this format.`
- `packages/aws/changelog.yml:12-16` documents this as version `6.16.0` behavior change.
**Documentation impact:**
- `packages/aws/docs/vpcflow.md` documents supported formats (`:51-62`) and states only that Parquet is unsupported (`:86`), but does not explain the new unsupported plain-text failure behavior.
- Users now need explicit guidance that unsupported plain-text formats generate `pipeline_error` events and that custom ingest pipelines are required for those layouts.
## Suggested Actions
- [ ] Update `packages/aws/docs/vpcflow.md` to document that unsupported plain-text VPC Flow layouts now trigger pipeline errors instead of being silently indexed.
- [ ] Add a short troubleshooting note describing expected error behavior and recommending custom ingest pipelines for unsupported formats.
- [ ] If applicable, add an example of an unsupported custom layout symptom (`pipeline_error` with unsupported format message) so users can recognize this condition quickly.
> [!NOTE]
>
> 🔒 Integrity filter blocked 37 items
>
> The following items were blocked because they don't meet the GitHub integrity level.
>
> - [#18066](https://github.com/elastic/integrations/issues/18066) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#18164](https://github.com/elastic/integrations/issues/18164) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#19124](https://github.com/elastic/integrations/pull/19124) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#18498](https://github.com/elastic/integrations/pull/18498) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#17506](https://github.com/elastic/integrations/pull/17506) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#19140](https://github.com/elastic/integrations/pull/19140) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#18671](https://github.com/elastic/integrations/pull/18671) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#19034](https://github.com/elastic/integrations/pull/19034) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#19061](https://github.com/elastic/integrations/pull/19061) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#18621](https://github.com/elastic/integrations/pull/18621) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#18746](https://github.com/elastic/integrations/pull/18746) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#18955](https://github.com/elastic/integrations/pull/18955) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#18944](https://github.com/elastic/integrations/pull/18944) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#18765](https://github.com/elastic/integrations/pull/18765) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#18951](https://github.com/elastic/integrations/pull/18951) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#18417](https://github.com/elastic/integrations/pull/18417) `search_pull_requests`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - ... and 21 more items
>
> To allow these resources, lower `min-integrity` in your GitHub frontmatter:
>
> ```yaml
> tools:
> github:
> min-integrity: approved # merged | approved | unapproved | none
> ```
>
>
---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Docs Patrol](https://github.com/elastic/integrations/actions/runs/26283774767)
Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on May 29, 2026, 11:06 AM UTC
Contributor guide
Assessment
This issue has not been assessed yet.