apache / apache/airflow

AirflowClusterPolicyViolation Overhaul - Pause DAG, Make it so that DAG cannot be unpaused, still shows in DAG list view but has tooltip saying why can't be unpaused

Open
#18,410 6 comments 1 reaction 0 assignees View on GitHub
kind:feature
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Description

## Overview
AirflowClusterPolicyViolation behaves in very strange ways right now. I think that this would be an important feature to overhaul for users that have complex DAG definition requirements for tagging, owners, naming conventions, etc. Very useful for Airflow administrators that have many application teams using the same deployment of Airflow.

## Current behavior
The AirflowClusterPolicyViolation exception when called from `airflow_local_settings.py` is that the DAG shows up still in the DAG list view (different from other import errors, where sometimes they will not show in this list any more). The DAG remains paused or unpaused depending on what state it was in before the DAG cluster policy was deployed. The DAG can still be scheduled and run, but all of the tasks within that DAG will fail with no errors and no logs. This silent failure is very confusing for developers that don't see the import error on their DAG.

## New Expected Behavior and Overhaul
The behavior that I expect to see when the AirflowClusterPolicyViolation is thrown is that the DAG will become paused, and cannot be unpaused until the DAG adheres to the cluster policy. There will be a tooltip on the pause button that explains why the DAG cannot be unpaused. The import error will show in the DAG view as well as the DAG list view, solved by https://github.com/apache/airflow/pull/17818. No tasks will be scheduled or run, and no DAGRuns will be scheduled or run, until the cluster policy is adhered to.

### Use case/motivation

I want to offer more support to users that have many application teams working on the same deployment of Airflow. Part of data pipeline quality is making policies that teams are unable to violate. If the teams adhere to the policy, they can run their DAG.

Right now the behavior is very confusing and it's challenging to pause a DAG that violates a cluster policy. It is possible to pause a DAG within the cluster policy function before the exception is raised, but there exists a race condition if the DAG is unpaused again after that. The DAG will try to run its tasks between the time that the DAG is unpaused and the DAG policy function has time to pause it again. This creates even more confusing behavior.

I believe that a DAG should not be able to be run unless it adheres to the cluster policy.

Very open to discussion about the best way to solve this issue.

### Related issues

_No response_

### Are you willing to submit a PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing how AirflowClusterPolicyViolation is handled from airflow_local_settings.py, including the DAG list and DAG views, and read PR #17818 for the related import-error display behavior. The change is complete when violating DAGs are paused, cannot be unpaused, show the policy reason in the pause-button tooltip, and schedule neither tasks nor DAG runs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.