aws / aws/aws-cdk

batch: Cannot omit the start/end of target node on batch.MultiNodeContainer

Open
#29,415 4 comments 0 reactions 0 assignees View on GitHub
@aws-cdk/aws-batch bug effort/medium p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
1d 19h
Merged PRs (30d)
71

Description

### Describe the feature

When using CDK to deploy aws batch multi-node jobs, we have to specify both the start_node and end_node, FYI: https://docs.aws.amazon.com/cdk/api/v2/python/aws_cdk.aws_batch/MultiNodeContainer.html#aws_cdk.aws_batch.MultiNodeContainer. But refer to the https://docs.aws.amazon.com/batch/latest/APIReference/API_NodeRangeProperty.html#API_NodeRangeProperty_Contents, it is possible to omit start_node/end_node here.

### Use Case

Our use case is as following: after I deploy the aws batch cloud infrastructure, I can use boto3 with python to submit a job like:

```
import boto3

response = batch_client.submit_job(
jobName=job_name,
jobQueue=job_queue,
jobDefinition=multi_job_definition,
parameters=job_parameters,
nodeOverrides={
"numNodes": overridden_num_nodes,
},
)
```

Currently It will throw error like:
```
botocore.errorfactory.ClientException: An error occurred (ClientException) when calling the SubmitJob operation: NumNodes override can only be applied if the job definition has at least 1 target node without a range_end i.e (:) or (range_start:).
```

This is because in CDK we have to specify the start_node and end_node. But if we support to omit the end_node in CDK, we can avoid this problem. And this is valid according to the aws batch multi-node job definition. Currently the only work around is to create another job_definition based on what we deploy and modify the target node in the batch console.

### Proposed Solution

_No response_

### Other Information

_No response_

### Acknowledgements

- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### CDK version used

2.131.0

### Environment details (OS name and version, etc.)

Amazon Linux 2

Contributor guide

Open the contributing guide

Research direction

Start at the AWS CDK aws_batch MultiNodeContainer construct and compare its start_node/end_node handling with AWS Batch's NodeRangeProperty documentation. Verify how a multi-node job definition is synthesized when either boundary is omitted, then add or update the relevant construct tests so the numNodes override scenario is covered and the generated definition accepts an open-ended target range.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.