elastic / elastic/package-spec

[Change Proposal] Pipeline benchmark test type

Open
#146 2 comments 1 reaction 0 assignees View on GitHub
discuss
Dominant language
Go
Stars
20
Forks
93
Avg merge
15h 10m
Merged PRs (30d)
12

Description

I would like to be able to benchmark a data stream's Elasticsearch ingest pipeline in order to measure throughput in events per second (EPS). The two uses cases for me would be

- for measuring the impact of changes to the pipeline (compare the "before" and "after" rates), and
- measuring throughput rates for a given hardware configuration over time (as a point of reference).

I can see this being implemented as either its own test type like `aws/data_stream/cloudtrail/_dev/test/benchmark` or as a feature of the current pipeline tests (like `benchmark: true`). In either case the test would provide log samples in the same formats as the current pipeline tests. These log samples would be sent repeatedly until 10k messages were processed. Then the tests would grab node stats metrics and analyze the results (like `GET _nodes/stats?metric=ingest&filter_path=nodes.*.ingest.pipelines`).

When executing the tests it would be useful to be able to test with a larger number or messages. For example:

`elastic-package test pipeline --benchmark --benchmark-messages=100000`

The result could be a simple table like:

| Test | Documents | Total Pipeline Time (ms) | EPS |
|--------------- |----------- |-------------------------- |----- |
| firewall-logs | 10000 | 12345 | 810 |

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.