aws-samples / aws-samples/sample-spot-balancer-spark-eks
patch_selector incorrectly adds workload role label as a nodeSelector
- Dominant language
- Python
- Stars
- 9
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
**Description:**
The patch_selector function in helpers.py adds both the capacity type label AND the workload role label (spark-role: executor) to the pod's nodeSelector. The workload role label is a pod-level identifier — it should not be used as a node selector because nodes don't have this label.
This causes executor pods patched with on-demand capacity to fail scheduling with:
`Failed to schedule pod, incompatible requirements, label "spark-role" does not have known values`
**Affected code (helpers.py):**
```
def patch_selector(settings: Any, capacity: str) -> list[dict[str, Any]]:
return [
{
"op": "add",
"path": "/spec/nodeSelector",
"value": {
settings.role_label: settings.executor_role_value, # ← BUG
settings.capacity_type_label: capacity,
},
}
]
```
**Expected behavior:**
The patch should only inject the capacity type label into the nodeSelector:
```
def patch_selector(settings: Any, capacity: str) -> list[dict[str, Any]]:
return [
{
"op": "add",
"path": "/spec/nodeSelector",
"value": {
settings.capacity_type_label: capacity,
},
}
]
```
**Impact:**
Pods assigned to on-demand by the webhook cannot be scheduled
Pods assigned to spot are unaffected (no patch applied in BEST_EFFORT mode)
The net result is that the spot ratio is never achieved — on-demand executors stay Pending indefinitely
Contributor guide
Research direction
Start in helpers.py at patch_selector and inspect how its JSON patch is used for executor pods. Remove the workload role entry from the nodeSelector so only the capacity type label is injected. Done means on-demand pods no longer include spark-role in nodeSelector while spot behavior remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- infrastructure
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 75/100