apache / apache/airflow

Option to "Disable Asset Event Generation" when running backfill

Open
#52,665 7 comments 0 reactions 0 assignees View on GitHub
area:backfill kind:feature needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Description

When running a backfill, provide the option (in the form of something like a checkbox) to "Disable Asset Event Generation" for the DAG being backfilled.

### Use case/motivation

For teams using Asset-driven scheduling to create "dependencies" between DAGs, there may arise a scenario where `DAG A` outlets to an `Asset 1`, which generates an Asset Event that schedules `DAG B`. However, if a user would like to backfill `DAG A` for X runs, this would result in X Asset Events being generated that would schedule `DAG B` X number of times.

This behavior may not be desired, and is something that's already been raised by a number of Airflow users.

### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

No files or tests are named in the issue. Start by locating the backfill entry point and the code that generates asset events, then trace how a backfill option could reach that behavior. Done means a backfill can disable asset event generation for the selected DAG without changing the default behavior, with coverage for the option and its effect.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.