apache / apache/airflow

Optimise asset-fetching in task

Open
#47,899 2 comments 0 reactions 1 assignee Claimed by @uranusjr View on GitHub
area:data-aware-scheduling kind:feature kind:meta priority:low
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Body

Currently inlet resolution (in `_AssetMainOperator`), `InletEventsAccessors`, and `OutEventsAccessors` all implement their own asset resolution logic. This is not only inefficient, but also a lot of duplication. We should try to put the logic in one place and add some simple cache.

### Committer

- [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.