aws / aws/containers-roadmap

[Batch] [request]: add array size environment variable

Open
#1,631 0 comments 53 reactions 0 assignees View on GitHub
Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**

Add an `AWS_BATCH_JOB_ARRAY_SIZE` enviornment variable to Batch array jobs.

**Which service(s) is this request for?**

Batch

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**

Array jobs receive an `AWS_BATCH_JOB_ARRAY_INDEX` variable to distinguish themselves in the array. However, this information alone is often not sufficient to distinguish an array worker's chunk of data on which they should operate.

Suppose we know in advance that there are 100 data items to process in the same manner, and we'd like to divide up this work using an array job. A single array worker's chunk of data to process depends on the total array size used. If the array size is 2, then worker 0 should process items 1-50, and worker 1 process 51-100. If the array size is 4, then worker 0 will only operate on items 1-25, etc.

**Are you currently working around this issue?**

Yes, by passing the array size as a CLI argument. However, the parity between the array size CLI input and the actual array size of the array job is only guarnteed through the business logic on job submission.

Contributor guide

Open the contributing guide

Research direction

The request concerns AWS Batch array jobs and exposing an AWS_BATCH_JOB_ARRAY_SIZE environment variable alongside AWS_BATCH_JOB_ARRAY_INDEX. No repository files, tests, or implementation entry points are identified; done would mean the variable reports the actual array size for each array worker.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.