elastic / elastic/integrations

Improve compatibility matrix testing

Open
#2,134 5 comments 1 reaction 0 assignees View on GitHub
Team:Ecosystem
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
2d 18h
Merged PRs (30d)
182

Description

Currently, packages in this repository have limited testing coverage with different versions of the stack:
* On PRs, they are tested with the last supported version according to the `kibana.version` constraint. This is done to ensure that backwards compatibility is warranted as defined in the package.
* CI jon can be parameterized with a `stackVersion`. There is a daily job testing with the last 7.x and 8.x snapshots, to know if some package breaks with new versions.

This has some problems:
* Upper bounds of the `kibana.version` constraints are not checked at any moment.
* Intermediary versions are not checked at any moment.
* Daily jobs attempt to run tests for all packages for the given version, even if the package doesn't support this version.

Some things to do:
* Ensure that at some moment at least the oldest and the newest supported version of a package is checked.
- [ ] https://github.com/elastic/integrations/issues/6072
* Fix CI jobs so they don't try to test a package with a version it doesn't support, when the job is launched for an specific `stackVersion`.
- [x] https://github.com/elastic/integrations/issues/2144
* Or, consider having a new daily job that tests packages with their supported matrix.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.