gchq / gchq/sleeper

ECSStateStoreCommitterThroughputST intermittent failure

Open
#7,601 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
107
Forks
29
Avg merge
19h 46m
Merged PRs (30d)
141

Description

### Description / Background

On 30/06/2026 the test ECSStateStoreCommitterThroughputST shouldMeetExpectedThroughputWhenCommittingFilesWithNoJobOnOneTable failed on the expectedCommitsPerSecondForTransactionLogOnly check as:
Expecting actual:
25.161663689203127
to be greater than:
100.0

We haven't see this failure again but the logs are available in the nightly system tests s3 bucket.

This test needs looking into to see what could have caused this.

Avenues to investigate:
Did the test experience a cold start which caused this?
Is there a problem with the implementation of this test in comparison to the other ECSStateStoreCommitterThroughput tests?
Is there an issue with how the logs are processed?

Contributor guide

Open the contributing guide

Research direction

Start with ECSStateStoreCommitterThroughputST, especially shouldMeetExpectedThroughputWhenCommittingFilesWithNoJobOnOneTable, and compare it with the other ECSStateStoreCommitterThroughput tests. Inspect the available logs in the nightly system tests S3 bucket and trace the expectedCommitsPerSecondForTransactionLogOnly check. Done means identifying whether the failure came from a cold start, test implementation, or log processing and recording or addressing the cause.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.