apache / apache/airflow

Extend DELETE API endpoints to support partitioned asset events

Open
#70,444 4 comments 0 reactions 0 assignees View on GitHub
area:API area:core kind:feature
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Description

On Airflow 3.3, the following endpoints:

DELETE /api/v2/assets/{asset_id}/queuedEvents
DELETE /api/v2/dags/{dag_id}/assets/queuedEvents
DELETE /api/v2/dags/{dag_id}/assets/{asset_id}/queuedEvents

only support deletion from AssetDagRunQueue but not PartitionedAssetKeyLog or AssetPartitionDagRun.

I think these should support an optional "partition_key" parameter to mirror behaviour from POST /api/v2/assets/{asset_id}/materialize.

### Use case/motivation

Sometimes a partitioned event is incorrectly emitted or no longer relevant for triggering downstream processes, for example:

Data is later determined to be invalid.
The downstream has been triggered by a more recent partition.
Dependency /partition logic has been updated after the partitioned event has been registered.

In such cases, it will be useful to delete the irrelevant partitioned event so that it doesn't trigger unintentional downstreams and show up as a stale asset in the UI.

### Related issues

_No response_

### Are you willing to submit a PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by tracing the three DELETE endpoints listed in the issue and compare their parameter handling with POST /api/v2/assets/{asset_id}/materialize. Inspect how AssetDagRunQueue, PartitionedAssetKeyLog, and AssetPartitionDagRun are affected. Done means an optional partition_key supports deleting the relevant partitioned event through each applicable endpoint, with API tests covering the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.