apache / apache/airflow

Deprecate Asset.extra and add Asset.event_extra_template

Open
#55,200 13 comments 1 reaction 1 assignee Claimed by @uranusjr View on GitHub
area:core area:data-aware-scheduling kind:feature priority:medium
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Body

`Asset.extra` was originally designed so users can attach additional useful information to describe an asset. We also designed `AssetEvent.extra` so when an event is emitted against an asset, it can contain additional data to be used in dags/tasks that are triggered by the event, or read the asset as a data inlet.

However, perhaps due to the very similar name, `Asset.extra` is constantly being misunderstood as some form of blueprint/template/source to `AssetEvent.extra`, and key-values defined on the asset should be populated into event extras against the asset.

Combined with the latest attempt to add extra templating (#54885 by @hussein-awala), I feel the confusion is much too significant for this feature. My proposal for the situation is

1. Deprecate `Asset.extra` to be removed in 4.0. It is not clear whether a key-value store for static additional information is useful. Dags don’t have it, and people don’t seem to be too bothered. We can probably add `doc_md` etc. if needed.
2. Add `Asset.event_extra_template` to re-implement #54885. Clearer name, no backward compatibility consideration needed.

### Committer

- [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.