elastic / elastic/elastic-package

[Feature request] Enable flags for total_events rally benchmarking paramaters

Open
#1,699 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
72
Forks
141
Avg merge
19h 42m
Merged PRs (30d)
55

Description

## Description

The configuration for `elastic-package` benchmarks are stored in YAML. Some examples of how these files are defined are found [here](https://github.com/elastic/elastic-package/blob/main/docs/howto/rally_benchmarking.md) and I've also copied one below:

```yaml
---
---
description: Benchmark 20000 events ingested
data_stream:
name: testds
corpora:
generator:
total_events: 900000
template:
type: gotext
path: ./logs-benchmark/template.ndjson
config:
path: ./logs-benchmark/config.yml
fields:
path: ./logs-benchmark/fields.yml
```

What we want to be able to do is to quickly run comparative benchmarks for an integration using a test harness which calls `elastic-package` directly. At present, here are the configuration options for `elastic-package` when called in benchmark mode:

```
Run rally benchmarks for the package (esrally needs to be installed in the path of the system)

Usage:
elastic-package benchmark rally [flags]

Flags:
--benchmark string name of the benchmark scenario to run
--defer-cleanup duration defer test cleanup for debugging purposes
--dry-run do not run rally but just generate the rally track
-h, --help help for rally
--metrics-collection-interval duration the interval at which metrics are collected (default 1s)
--package-from-registry string fetch package from registry instead of local directory, expected format: -
--rally-track-output-dir string output dir of the rally track: if present the command will save the generated rally track
--reindex-to-metricstore if set the documents from the benchmark will be reindexed to the metricstore for posterior analysis
--use-corpus-at-path string path of the corpus to use for the benchmark: if present no new corpus will be generated
--variant string service variant

Global Flags:
-p, --profile string select a profile to use for the stack configuration. Can also be set with ELASTIC_PACKAGE_PROFILE
-v, --verbose verbose mode
```

What we would like to see added here are two flags:

|Flag|Description|
|----|------------|
|`--total-events`| When passed would supersede the `total_events` value in the configuration YAML|
|`--data-stream`|When passed would supersede the `name` value under the `data_stream` key. |

Having this will greatly aid us in being able to use `elastic-package` for ad-hoc and automated performance testing.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the `elastic-package benchmark rally` command and the benchmark configuration described in `docs/howto/rally_benchmarking.md`. Add `--total-events` and `--data-stream` so passed values supersede the YAML settings, then verify the flags appear in the command help and affect generated benchmark runs.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, performance
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.