elastic / elastic/apm-server

[META] Benchmark 2.0: Write benchmark intake event data upload tool

Open
#7,844 1 comment 0 reactions 0 assignees View on GitHub
benchmarking enhancement
Dominant language
Go
Stars
1.3k
Forks
543
Avg merge
1d 18h
Merged PRs (30d)
109

Description

## Description

We would like to stop committing the generated event data recorded by the [`intake-receiver`](https://github.com/elastic/apm-server/tree/main/cmd/intake-receiver) that is consumed by `apmbench` to the main `apm-server` repository. Currently the events are stored in [`systemtest/benchtest/events`](https://github.com/elastic/apm-server/tree/main/systemtest/benchtest/events)for convenience, but it is not ideal to commit such large files to the git repository, particularly as we add more datasets from different agents and evolve the generated data set.

The main goal of this tool is to allow uploading and downloading the events from where they're stored (most likely we should look into leveraging blob storage services from cloud providers) and perhaps filter which files are downloaded, for example, allow downloading only a single language agent's benchmark (such as `go-2.0.0.ndjson`), and which version of the agent should be downloaded (latest or a specific revision).

We should also explore if and how we'd like to some versioning on the files that are uploaded.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.