[feature request] Allow OneZone EFS with Multi-AZ Compute Node Configuration
- Dominant language
- Python
- Stars
- 888
- Forks
- 314
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 43
Description
## Problem Statement
We're experiencing increasing InsufficientCapacity errors when launching compute nodes. As a workaround, we've implemented multiple compute node subnets across availability zones to allow ParallelCluster to retry provisioning in another AZ when capacity is unavailable.
However, we're unable to combine this multi-AZ compute approach with EFS OneZone storage, which offers better performance for many use cases (even when mounted across zones). Tools like Posit Workbench recommend OneZone for performance benefits. (https://docs.posit.co/posit-team/index.html)
## Current Limitation
ParallelCluster validation prevents configuring OneZone EFS with compute nodes spanning multiple AZs, showing the error:
> "EFS OneZone is only supported if all compute nodes and the head node are in the same Availability Zone. EFS OneZone can have only one mount target."
## Requested Enhancement
Please modify ParallelCluster to:
1. Allow the configuration of EFS OneZone storage with compute nodes distributed across multiple AZs
2. Ideally, prioritize launching compute nodes in the same AZ as the EFS OneZone storage
3. Fall back to other AZs only when capacity isn't available in the primary AZ
This change would maintain the performance benefits of OneZone EFS while adding resilience against capacity constraints.
Contributor guide
Research direction
Start by tracing the validation that emits “EFS OneZone is only supported if all compute nodes and the head node are in the same Availability Zone.” Review how compute-node subnets and EFS storage placement are represented, then determine how same-AZ preference and cross-AZ fallback should be specified. Done means a supported configuration can validate and provision with the requested behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100