Performance benchmark as a CI step
- Dominant language
- No language data
- Stars
- 126
- Forks
- 26
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe.**
Currently unit, integration and functional tests are triggered during the CI build on Azure DevOps.
Impact of new features is not monitored automatically. It would be very beneficial, to setup some performance benchmark as a part of the pipeline that will run simple performance check and compare the results agains some threshold.
**Describe the solution you'd like**
Performance benchmark is a part of the build pipeline.
The very basic idea would be to run the knotx-starterkit as a docker container and use some load generator ([this](https://k6.io/blog/integrating-load-testing-with-azure-pipelines) looks quite promising) and gather results.
Additionally there should be threshold (KPI) set that will fail the build if KPI is not met.
**Describe alternatives you've considered**
There are a few alternatives:
- instead of knoxt-starterkit use some knotx-examples to cover more complex scenarios in tests
- performance checks are scheduled nightly not as parto fo the build pipeline
**Additional context**
As part of this task, some Azure DevOps RnD should be done, to check if that's possible to run such performance benchmark during the pipeline and to check if any additional servers to run Knot.x are required or not.
Contributor guide
Research direction
Start by investigating the Azure DevOps pipeline and the linked k6 integration example. Evaluate running knotx-starterkit in Docker, define a KPI threshold, and determine whether build-time or nightly execution and additional servers are needed. Done means the benchmark runs in the chosen pipeline mode and fails when the KPI is not met.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, docker
- Domain
- ci-cd, devops, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100