elastic / elastic/package-spec

Improve support for transforms

Open
#370 7 comments 4 reactions 0 assignees View on GitHub
Team:Ecosystem
Dominant language
Go
Stars
20
Forks
93
Avg merge
15h 10m
Merged PRs (30d)
12

Description

We merged basic transforms support in https://github.com/elastic/package-spec/pull/307, but there are several outstanding and related problems that remain to be solved.

### Short-term - hard blockers for next security packages
- [x] https://github.com/elastic/package-spec/pull/307
- [x] https://github.com/elastic/kibana/issues/134321
* This implements the package-spec definition above so transforms in packages can actually be installed. Until this is finished, packages can contain transforms, but they will not be installed.
- [x] https://github.com/elastic/kibana/issues/137278
* `kibana_system` is currently used to install all transforms. This is necessary in the Endpoint case where the package must be upgraded along with Kibana. CSP may need this too.
* The problem with this is that it requires `kibana_system` to be granted read/write privileges to transform src/dest indices

### Medium-term - likely not hard blockers for next security packages
- [ ] Better upgrade strategy for transforms
* Rebuilding transform destination indices can be very expensive, but is necessary in some cases
* Should avoid it in obvious upgrade scenarios (eg. a visualization was updated)
* Updating the frequency or delay is supported by the transform API, Fleet could leverage this to avoid re-building transforms unnecessarily so.
* Will definitely need to destroy and re-create a transform when the aggregation or fields have changed. This could be improved, but likely lower priority at this time.
- [ ] https://github.com/elastic/package-spec/issues/202

### Long-term
- [ ] #460
- [ ] Enforce the data stream naming convention for transform src and dest indices

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.