gchq / gchq/sleeper

Performance test bulk import on EMR Serverless

Open
#7,421 0 comments 0 reactions 0 assignees View on GitHub
system-test-module
Dominant language
Java
Stars
107
Forks
29
Avg merge
19h 46m
Merged PRs (30d)
141

Description

### User Story

As a developer of Sleeper, I want to know when performance of bulk import on EMR Serverless degrades, so that I can resolve it if it's caused by a change in Sleeper.

As a developer of Sleeper, I want to know if there's a problem in bulk import on EMR Serverless that only shows up with a large enough job, so that I can fix it.

### Description / Background

We'd like a performance test for bulk import on EMR Serverless.

We probably don't want this until until we retain the logs from EMR after a nightly system test suite (https://github.com/gchq/sleeper/issues/5912), otherwise we wouldn't be able to maintain it.

### Technical Notes / Implementation Details

This should be similar to the existing EksBulkImportPerformanceST.

### Dependencies / Blockers

Depends on:
- https://github.com/gchq/sleeper/issues/5912

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.