apache / apache/airflow

Simple extra link functionality when leveraging TaskFlow

Open
#54,527 13 comments 0 reactions 0 assignees View on GitHub
area:core kind:feature
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Description

Enable easy setting of a extra link(s) within a task-decorated/TaskFlow function. Currently one must create an custom operator for this functionality and define an extra link class, and thus extra links are not utilized for some one-off tasks by DAG authors where they could be very useful and provide value.

### Use case/motivation

Provide a common and easy setting of extra links within task-decorated functions that enables DAG authors to:
- Very easily add extra link(s) to their new or pre-existing task decorated functions.
- Support for multiple extra links for a single function
- Does not require altering the return value of the function
- Overrides any extra links w/same name for this task instance

For example as a separate decorator:
```
@with_extra_links("API Resource")
@task
def get_api_resource(id_: str):
"""
Example of setting extra link w/a separate decorator with link name `API Resource`.
"""
rsp = requests.get(f"https://localhost/v2/api/fakeresource/{id_}")
set_extra_link("API Resource", rsp.url)
return rsp.json()
```

Or adjusting the existing decorator:
```
@task(extra_links=["API Resource"])
def get_api_resource(id_: str):
"""
Example of setting extra link w/a separate decorator with link name `API Resource`.
"""
rsp = requests.get(f"https://localhost/v2/api/fakeresource/{id_}")
set_extra_link("API Resource", rsp.url)
return rsp.json()

```

### Related issues

_No response_

### Are you willing to submit a PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by tracing the TaskFlow @task decorator and the set_extra_link call shown in the issue, then compare the separate @with_extra_links("API Resource") approach with the proposed task(extra_links=[...]) form. Check how decorated functions expose task-instance context and how existing extra links are resolved. Done means authors can add multiple links without changing return values, while same-name links override existing ones.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering, developer-experience
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.