elastic / elastic/elastic-package
[Feature request] Enable flags for total_events rally benchmarking paramaters
- Dominant language
- Go
- Stars
- 72
- Forks
- 141
- Avg merge
- 19h 42m
- Merged PRs (30d)
- 55
Description
## Description
The configuration for `elastic-package` benchmarks are stored in YAML. Some examples of how these files are defined are found [here](https://github.com/elastic/elastic-package/blob/main/docs/howto/rally_benchmarking.md) and I've also copied one below:
```yaml
---
---
description: Benchmark 20000 events ingested
data_stream:
name: testds
corpora:
generator:
total_events: 900000
template:
type: gotext
path: ./logs-benchmark/template.ndjson
config:
path: ./logs-benchmark/config.yml
fields:
path: ./logs-benchmark/fields.yml
```
What we want to be able to do is to quickly run comparative benchmarks for an integration using a test harness which calls `elastic-package` directly. At present, here are the configuration options for `elastic-package` when called in benchmark mode:
```
Run rally benchmarks for the package (esrally needs to be installed in the path of the system)
Usage:
elastic-package benchmark rally [flags]
Flags:
--benchmark string name of the benchmark scenario to run
--defer-cleanup duration defer test cleanup for debugging purposes
--dry-run do not run rally but just generate the rally track
-h, --help help for rally
--metrics-collection-interval duration the interval at which metrics are collected (default 1s)
--package-from-registry string fetch package from registry instead of local directory, expected format: -
--rally-track-output-dir string output dir of the rally track: if present the command will save the generated rally track
--reindex-to-metricstore if set the documents from the benchmark will be reindexed to the metricstore for posterior analysis
--use-corpus-at-path string path of the corpus to use for the benchmark: if present no new corpus will be generated
--variant string service variant
Global Flags:
-p, --profile string select a profile to use for the stack configuration. Can also be set with ELASTIC_PACKAGE_PROFILE
-v, --verbose verbose mode
```
What we would like to see added here are two flags:
|Flag|Description|
|----|------------|
|`--total-events`| When passed would supersede the `total_events` value in the configuration YAML|
|`--data-stream`|When passed would supersede the `name` value under the `data_stream` key. |
Having this will greatly aid us in being able to use `elastic-package` for ad-hoc and automated performance testing.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the `elastic-package benchmark rally` command and the benchmark configuration described in `docs/howto/rally_benchmarking.md`. Add `--total-events` and `--data-stream` so passed values supersede the YAML settings, then verify the flags appear in the command help and affect generated benchmark runs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli, performance
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100