aws-samples / aws-samples/sample-spot-balancer-spark-eks
SpoBalancer pod not starting properly
- Dominant language
- Python
- Stars
- 9
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I've followed instructions to deploy spot-balancer on EKS AutoMode Cluster and when I'm checking spot-balancer-webhook logs, I see following errors. Any idea what this can be related to?
```shell
kubectl -n spark-tools logs deploy/spot-balancer-webhook -c spot-balancer --tail=100
AwsEksResourceDetector failed: HTTP Error 403: Forbidden
AwsEc2ResourceDetector failed: timed out
[2026-05-22 03:45:08 +0000] [1] [INFO] Starting gunicorn 26.0.0
[2026-05-22 03:45:08 +0000] [1] [INFO] Listening at: https://0.0.0.0:8443 (1)
[2026-05-22 03:45:08 +0000] [1] [INFO] Using worker: sync
[2026-05-22 03:45:08 +0000] [21] [INFO] Booting worker with pid: 21
[2026-05-22 03:45:08 +0000] [1] [ERROR] Control server error: [Errno 30] Read-only file system: '/home/appuser/.gunicorn'
2026-05-22 03:45:08,693 INFO Starting reconcile of Redis state from K8s
2026-05-22 03:45:08,693 INFO Reconcile worker started (interval=1800s)
2026-05-22 03:45:08,704 ERROR Reconcile error: (401)
Reason: Unauthorized
HTTP response headers: HTTPHeaderDict({'Audit-Id': '449ca280-3a06-4ef7-b1ad-9292204780ca', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Date': 'Fri, 22 May 2026 03:45:08 GMT', 'Content-Length': '129'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401}
```
Contributor guide
Research direction
Start by reproducing the EKS AutoMode deployment and reviewing the spot-balancer-webhook logs shown in the issue. Investigate the reported AWS detector permissions, Kubernetes 401 response, and read-only /home/appuser/.gunicorn path, then identify the configuration or permissions change that allows the pod to start and reconcile Redis state successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes, python, redis
- Domain
- authentication, cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100