gchq / gchq/sleeper

Handle visibility timeout and retries in persistent state store committer

Open
#6,464 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
107
Forks
29
Avg merge
19h 46m
Merged PRs (30d)
141

Description

### Description / Background

Follows on from:
- https://github.com/gchq/sleeper/issues/5091

In the linked issue we wrote code to receive state store commit requests from SQS, and we set a hard coded visibility timeout of 15 minutes. That means if the committer takes a long time on a batch of commits, or it crashes, then after 15 minutes those commits will reappear on the queue to be processed again.

That's a very long time for the commits to be left unprocessed if the committer crashes. We'd like a shorter visibility timeout, and a keep alive thread similar to the ones in ingest and compaction, to prevent it timing out if the committer is slow.

There's also a TODO for handling retries.

### Technical Notes / Implementation Details

There are two TODO comments for this in the multithreaded committer code.

Ingest and compaction both use PeriodicActionRunnable with a ChangeMessageVisibilityTimeoutAction from a MessageReference.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.