open-telemetry / open-telemetry/opentelemetry-python

Add benchmarks to every type of package in `build` workflow

Open
#1,468 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

build & infra triaged
Dominant language
Python
Stars
2.6k
Forks
1k
Avg merge
4d 15h
Merged PRs (30d)
19

Description

In #1443 we add performance tests for the SDK.

The SDK falls under the category core of tests matched in the build workflow. For this reason, when the jq command looks through the entire repo after the workflow for *-benchmark.json files we are guaranteed to find at least ` file. This file is then given to the subsequent Github action to report on benchmarking results.

However, the build workflow also has instrumentation and exporter packages. These don't have performance tests yet so the jq command will fail. Even if we suppress the failure, the subsequent GitHub action will fail if no valid output.json file is created.

A temporary solution was to only run those 2 steps of the workflow if: matrix.package == 'core'. A more long-term solution will be to add benchmarks to instrumentation and exporter.

We know only these packages are required because as long as subsequent tests fit under these categories we won't have to keep adding if statements.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the build workflow and the existing SDK benchmarks added in #1443. Trace how jq discovers *-benchmark.json files and how output.json is passed to the reporting GitHub Action, then add equivalent performance tests for the instrumentation and exporter packages. Done means both package categories produce valid benchmark and output files without package-specific workflow conditions.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
ci-cd, performance, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.