open-telemetry / open-telemetry/opentelemetry-java-instrumentation

Test Telemetry Generator

Open
#14,161 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
2.6k
Forks
1.2k
Avg merge
2d 18h
Merged PRs (30d)
228

Description

As part of #13468, we are currently running a bash script in order to execute a specific list of gradle tasks with the -PcollectMetadata=true flag. This results in the generation of all the .telemetry files that we then scrape and analyze when we run the :instrumentation-docs:runAnalysis task to populate the instrumentation-list.yaml.

The reasons we need this script:

  • As we work out the mechanics of the telemetry generation, it is easier to start small and only enable and run the process for a small number of modules to make reviewing the results easier.
  • Running the entire test suite for every instrumentation module takes a really long time, and can be flaky on a local machine. Therefore at least for now, it makes sense to just run the modules we care about/have enabled things for.
  • Some test suites don't work on ARM architectures, and need to be run using colima instead of standard docker. This script separates those instrumentations and runs them differently in order to avoid failures when generating telemetry locally.

Once we have worked through and enabled and validated the telemetry emitted by all modules, the plan is to run this process on a schedule using github actions, and much of this complexity won't be needed.

In the interim, the bash script is getting a bit heavy. It would likely be better to orchestrate these gradle tasks another way. This issue is to investigate using a solution written in Python or using another approach.

Requirements:

  • Runs test suites with the -PcollectMetadata=true flag enabled
  • Supports running multiple suites per module (ex: test, testStableSemconv)
  • When generating telemetry, it clears out previously generated data so that the data is fresh
  • Ability to run some modules using a standard docker environment, but also be able to specify modules that should be run on colima x86_64
  • It would be nice to be able to specify particular modules to run the process for, instead of the entire list, since that takes a long time. As of now I have just been commenting out all modules except the ones I want to run it for, and that is cumbersome.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading instrumentation-docs/collect.sh and the :instrumentation-docs:runAnalysis task, with instrumentation-list.yaml as the generated-data context. Review how the script runs suites with -PcollectMetadata=true, clears telemetry data, handles Docker and Colima x86_64, and selects modules. Done means a maintainable approach supports the listed suite, environment, data-refresh, and module-selection requirements.

Written by the indexing model from the issue text.

Assessment

Tech stack
bash, docker, java
Domain
build-system, devops, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.