open-telemetry / open-telemetry/opentelemetry-java
Benchmark performance report published (and repeatable)
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 2.5k
- Forks
- 1k
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 58
Description
Is your feature request related to a problem? Please describe.
I'd like to very quickly be able to find performance characteristics of SDK components. The reasons are severalfold:
- I'd like to understand if the Google-Cloud exporters are on-par with OTLP performance
- I'd like to understand the CPU/Memory pressure caused via OTel's SDK and various mechanisms to alleviate it
- I'd like to understand if we should be promoting different defaults for different envirornments, e.g. K8s vs. K-Native may be highly different in terms of what recommended setup we should promote.
Those are just top-of-mind. In general, it'd be nice if performance characteristics were as public as posisble (without requiring one to run the entire suite).
Describe the solution you'd like
I'd like to update the JMH hooks in the gradle build with some kind of task that will collect and generate markdown reports on that state of any given PR.
The goal would be to (one day) extend the performance monitoring-github action to autopublish performance dumps and over-time analysis of this SDK.
However, I see it as a multi-step process:
- Create a task which pulls all JMH reports into one central report dump (can be multiple markdown files.
- Enhance the task in a series of PRs and discussions until we're happy with its format.
- Automate a GithubAction to generate performance reports when commits hit master (or every so often).
Note: The Python SDK already does this, and it's pretty nice.
Describe alternatives you've considered
Keeping things as is. Didn't really consider many alternatives so happy to hear ideas.
Additional context
Python configuration: https://github.com/open-telemetry/opentelemetry-python/blob/master/.github/workflows/test.yml#L72
Python example: https://open-telemetry.github.io/opentelemetry-python/benchmarks/
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the JMH hooks in the Gradle build and review the linked Python workflow and benchmark example for report-generation patterns. The first milestone is a task that gathers JMH reports into central Markdown files; the broader work also calls for format iteration and a GitHub Action for publishing reports.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, java
- Domain
- build-system, ci-cd, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100