apache / apache/airflow

Investigate if we can optimise register_asset_changes_in_db from task runner

Open
#46,002 0 comments 0 reactions 1 assignee Claimed by @amoghrajesh View on GitHub
area:data-aware-scheduling area:task-execution-interface-aip72
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Body

Right now, for registering asset events on task completion, we send across a lot of "events" for AssetNameRef and AssetUriRef types. Maybe we can optimise it by retrieving the Asset directly using client APIs and sending those.

This will also simplify `register_asset_changes_in_db`

### Committer

- [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.