elastic / elastic/package-spec

Proposal: Make IngestPipeline tag be able to reference other data_streams/packages pipelines

Open
#580 1 comment 0 reactions 0 assignees View on GitHub
discuss enhancement Team:Security-External Integrations
Dominant language
Go
Stars
20
Forks
93
Avg merge
15h 10m
Merged PRs (30d)
12

Description

We have some data streams that have duplicated pipelines because currently it is not possible to reference pipelines outside of the current data stream using the `IngestPipeline` tag.

Would be nice to extend it so it could be used in a similar way to:

- `{{ IngestPipeline nameOfPipeline }}` for pipelines in the same data stream
- `{{ IngestPipeline nameOfDatastream/nameOfPipeline }}` for pipelines in other data streams of the same package

Additionally, even though maybe not as big of a need right now, something like `{{ IngestPipeline nameOfPackage/versionOfPackage/nameOfDatastream/nameOfPipeline }}` could be used to be able to reference any pipeline from any package. The version must be specified if we want to reference pipelines across packages to avoid breaking changes.

Another consideration is that if we include a pipeline from another data stream or package, we need to not only install it but also any other pipelines that it references, and manage the life cycle of these dependencies accordingly.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.