Support multi-instance tasks on concurrency enabled pools
- Dominant language
- No language data
- Stars
- 27
- Forks
- 29
- PR merge metrics
- No merged PRs in 30d
Description
It's possible that I am not configuring my tasks correctly, because this seems like a feature that would already exist. But, I am having trouble getting RequiredSlots and multi-instance tasks to work.
When I create a batch job it contains thousands of tasks. I would say that 75% of the tasks created are small, and a single node could run 4 tasks concurrently. The remaining 25% are tasks that require many cores with MPI.
I had MPI working, but now need to implement the RequiredSlots feature in order to optimize our node usage.
I've refactored the small tasks to use RequiredSlots, and that seems to be working now. However, the MPI task that requires multiple nodes no longer executes.
The documentation states that the [requiredSlots property be set to 1](https://docs.microsoft.com/en-us/rest/api/batchservice/task/add) if it's a multi-instance task. I have tried setting this to 1, with the multi-instance property requiring 2 nodes, but the task never executes. I've also tried setting requiredSlots to the pool's TaskSlotsPerNode property (just to check), but of course received a validation exception stating the property value was invalid.
Are there any known issues with running a workflow that consists of tasks that consume just 1 slot (with each node having 4 slots), and other tasks that require multiple nodes with MPI?
Contributor guide
Assessment
This issue has not been assessed yet.