elastic / elastic/integrations
Improve compatibility matrix testing
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 182
Description
Currently, packages in this repository have limited testing coverage with different versions of the stack:
* On PRs, they are tested with the last supported version according to the `kibana.version` constraint. This is done to ensure that backwards compatibility is warranted as defined in the package.
* CI jon can be parameterized with a `stackVersion`. There is a daily job testing with the last 7.x and 8.x snapshots, to know if some package breaks with new versions.
This has some problems:
* Upper bounds of the `kibana.version` constraints are not checked at any moment.
* Intermediary versions are not checked at any moment.
* Daily jobs attempt to run tests for all packages for the given version, even if the package doesn't support this version.
Some things to do:
* Ensure that at some moment at least the oldest and the newest supported version of a package is checked.
- [ ] https://github.com/elastic/integrations/issues/6072
* Fix CI jobs so they don't try to test a package with a version it doesn't support, when the job is launched for an specific `stackVersion`.
- [x] https://github.com/elastic/integrations/issues/2144
* Or, consider having a new daily job that tests packages with their supported matrix.
Contributor guide
Assessment
This issue has not been assessed yet.