elastic / elastic/package-spec
Improve support for transforms
- Dominant language
- Go
- Stars
- 20
- Forks
- 93
- Avg merge
- 15h 10m
- Merged PRs (30d)
- 12
Description
We merged basic transforms support in https://github.com/elastic/package-spec/pull/307, but there are several outstanding and related problems that remain to be solved.
### Short-term - hard blockers for next security packages
- [x] https://github.com/elastic/package-spec/pull/307
- [x] https://github.com/elastic/kibana/issues/134321
* This implements the package-spec definition above so transforms in packages can actually be installed. Until this is finished, packages can contain transforms, but they will not be installed.
- [x] https://github.com/elastic/kibana/issues/137278
* `kibana_system` is currently used to install all transforms. This is necessary in the Endpoint case where the package must be upgraded along with Kibana. CSP may need this too.
* The problem with this is that it requires `kibana_system` to be granted read/write privileges to transform src/dest indices
### Medium-term - likely not hard blockers for next security packages
- [ ] Better upgrade strategy for transforms
* Rebuilding transform destination indices can be very expensive, but is necessary in some cases
* Should avoid it in obvious upgrade scenarios (eg. a visualization was updated)
* Updating the frequency or delay is supported by the transform API, Fleet could leverage this to avoid re-building transforms unnecessarily so.
* Will definitely need to destroy and re-create a transform when the aggregation or fields have changed. This could be improved, but likely lower priority at this time.
- [ ] https://github.com/elastic/package-spec/issues/202
### Long-term
- [ ] #460
- [ ] Enforce the data stream naming convention for transform src and dest indices
Contributor guide
Assessment
This issue has not been assessed yet.