nextflow-io / nextflow-io/nextflow
Support accelerator directive for Azure Batch
Nobody has claimed this yet.
- Dominant language
- Groovy
- Stars
- 3.5k
- Forks
- 811
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 61
Description
The accelerator directive is currently supported for AWS Batch, Google Batch, and Kubernetes. Azure Batch also has GPUs, so this directive should also work for Azure Batch.
Not sure yet how to implement it yet with the Azure Batch Java SDK. All I found so far is the GpuResource class. @abhi18av do you have any idea how we would support GPU requests here? I don't know if there are specific GPU-enabled instance types, or if we can just attach a GPU to any instance or what.
In any case, I think we can map the accelerator type to the GpuSku enum:
final accel = task.config.getAccelerator()
new GpuResource()
.withCount( accel.getRequest() )
.withType( GpuSku.fromString(accel.getType()) )
So an example request would look like:
accelerator 1, type: 'V100'
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the existing accelerator support for AWS Batch, Google Batch, and Kubernetes, then inspect how the Azure Batch Java SDK represents GPU resources and instance types. Determine whether GpuResource and GpuSku apply to Azure Batch, and define the supported mapping for an example such as accelerator 1, type: 'V100'. Done means Azure Batch accepts and provisions the requested accelerator consistently with the other backends.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, groovy
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100