elastic / elastic/integrations
[All Integrations] Users should be able to do custom processing after a pipeline failure.
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 225
Description
Hello,
Currently when an ingest pipeline processor fails for some reason and it does not have the option `ignore_failure` set to `true`, it will break the processing of further processors, including the custom pipeline for the integration.
When this happens the user has no way to do any processing on this event as all customization needs to be done on the `@custom` ingest pipeline, and this pipeline is skipped when a failure happens.
For example, I have an issue with the Fortigate pipeline where some devices send some events with the wrong format or truncated and this breaks the pipeline, this generate thousands of events per hour that I cannot drop using a custom ingest pipeline as it is ignored.
So the integrations should have a way for the user to capture those failed events and do something with them.
Contributor guide
Assessment
This issue has not been assessed yet.