gchq / gchq/sleeper

Bulk import jobs fail when capacity is unavailable for more EMR clusters

Open
#780 0 comments 0 reactions 0 assignees View on GitHub
bulk-import-module
Dominant language
Java
Stars
107
Forks
29
Avg merge
19h 46m
Merged PRs (30d)
141

Description

If an EMR cluster cannot be created, eg. due to insufficient capacity, bulk import jobs will fail. They are not added back to the bulk import queue, and they do not show up in any reports.

We could look at ways to manage this further:

https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-instance-fleet.html

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.