apache / apache/airflow

Resource quota aware kubernetes pod operator

Open
#63,944 2 comments 0 reactions 0 assignees View on GitHub
kind:feature needs-triage provider:cncf-kubernetes
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Description

If KubernetesPodOperators were resource quota aware the proper intent could be provided so the task could be queued/retried rather then always failing.

### Use case/motivation

Sometimes I will have too many dags kick off that spin up pods and I will hit my quota, rather then having my runs fail I would prefer if that task was just queued until the quota frees up as I know this will happen "soon".

### Related issues

_No response_

### Are you willing to submit a PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by locating the KubernetesPodOperator entry point and the code that handles pod-creation failures when a Kubernetes resource quota is exceeded. Define how quota exhaustion should signal queueing or retrying rather than task failure, and verify that runs wait for quota to free up without changing other pod failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, python
Domain
devops
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.