aws / aws/containers-roadmap

[Batch] [Error-tolerant sequential job processing]: processing a queue of jobs sequentially despite failed jobs

Open
#2,183 0 comments 0 reactions 0 assignees View on GitHub
Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**
What do you want us to build?
Support for batch processing a queue of jobs sequentially regardless of whether jobs fail or succeed.

**Which service(s) is this request for?**
Batch

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
What outcome are you trying to achieve, ultimately, and why is it hard/impossible to do right now? What is the impact of not having this problem solved? The more details you can provide, the better we'll be able to understand and solve the problem.
Currently, batch supports array jobs with the SEQUENTIAL dependency in order to execute a queue of jobs sequentially. However, the completion of the queue is dependent on every job being successful. A failed job, therefore, will prevent the remaining jobs from running. This feature request is for a way to be able to sequentially run a queue of jobs from beginning to end regardless of whether each job fails or succeeds.

**Are you currently working around this issue?**
How are you currently solving this problem?

**Additional context**
Anything else we should know?

**Attachments**
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

Contributor guide

Open the contributing guide

Research direction

No source files, tests, or implementation entry points are identified; this is a request for AWS Batch behavior. Start by reviewing AWS Batch array jobs and SEQUENTIAL dependency semantics, then define the failure-handling behavior and acceptance criteria for running the queue to completion.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.