open-telemetry / open-telemetry/opentelemetry-python
Add benchmarks to every type of package in `build` workflow
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 1k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 19
Description
In #1443 we add performance tests for the SDK.
The SDK falls under the category core of tests matched in the build workflow. For this reason, when the jq command looks through the entire repo after the workflow for *-benchmark.json files we are guaranteed to find at least ` file. This file is then given to the subsequent Github action to report on benchmarking results.
However, the build workflow also has instrumentation and exporter packages. These don't have performance tests yet so the jq command will fail. Even if we suppress the failure, the subsequent GitHub action will fail if no valid output.json file is created.
A temporary solution was to only run those 2 steps of the workflow if: matrix.package == 'core'. A more long-term solution will be to add benchmarks to instrumentation and exporter.
We know only these packages are required because as long as subsequent tests fit under these categories we won't have to keep adding if statements.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the build workflow and the existing SDK benchmarks added in #1443. Trace how jq discovers *-benchmark.json files and how output.json is passed to the reporting GitHub Action, then add equivalent performance tests for the instrumentation and exporter packages. Done means both package categories produce valid benchmark and output files without package-specific workflow conditions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, python
- Domain
- ci-cd, performance, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100