canonical / canonical/observability-stack
QA infra test improvements
- Dominant language
- HCL
- Stars
- 9
- Forks
- 12
- Avg merge
- 2d 40m
- Merged PRs (30d)
- 3
Description
For context, first read this HowTo:
- https://github.com/canonical/observability-team/blob/main/how-to/test-products-with-solqa.md
We should improve our QA tests so we have a maintainable implementation and better coverage. This can be done in a few ways:
- Add pytests to our `sqa` folder under [tests](https://github.com/canonical/observability-stack/tree/main/tests), and any other helper functions there. These are executed by the `sqa_tests` in the root repo.
- We should try to avoid having a test variant per product flavor: COS / COS Lite. The main differences are otelcol and timeouts
- We likely don't want to wait 2h for active/idle for COS Lite, although this is valid for COS.
- Consider a good design for checking which flavor we have and piping that into each helper script.
- We should choose which tests we already have in [tests/solution](https://github.com/canonical/observability-stack/tree/main/tests/integration) and port them over to `tests/sqa`, or better yet, make them reusable?
- We should output our results in xUnit format, like how [the Juju team](https://github.com/canonical/terraform-provider-juju-qa/blob/6f49238cfdc1ac3944ff13750f8c4076b2247376/sqa_tests#L52) does it. They do it in GoLang, but we can do it in some Pythonic way.
- This will get rid of [this warning](https://github.com/canonical/terragrunt-deployment-pipelines/actions/runs/28136732680/job/83325143528#step:30:236)
- We could also address [this warning](https://github.com/canonical/terragrunt-deployment-pipelines/actions/runs/28136732680/job/83325143528#step:30:265) by generating a `metrics.json` file (likely a stretch goal).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with how-to/test-products-with-solqa.md, then inspect the existing tests/sqa and tests/integration directories and the root repository's sqa_tests entry point. Compare current COS and COS Lite test variants and identify reusable helpers and coverage to port. Done should include a maintainable test design with appropriate flavor-specific behavior and xUnit output; metrics.json is identified as a stretch goal.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, testing
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100