apache / apache/airflow

Cannot set `operator_extra_links` in Operator `__init__` method for dynamically-mapped Tasks

Open
#54,778 7 comments 0 reactions 0 assignees View on GitHub
area:core area:dynamic-task-mapping kind:feature
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Apache Airflow version

`main`

### If "Other Airflow 2 version" selected, which one?

_No response_

### What happened?

When attempting to set `operator_extra_links` in the `__init__` method of an Operator, a button routing to that link will be properly displayed and functional. However, this is not the case when using dynamically mapped Tasks.

**Attempting to set `extra_operator_links` in the `__init__` method of an Operator that is eventually "dynamically-mapped" fails to render the button for each mapped Task instance.**

### What you think should happen instead?

When `operator_extra_links` are set in the `__init__` method of an Operator, the corresponding button should be rendered in the "Details" tab of each mapped Task instance.

### How to reproduce

I tested the following, with the noted results for each. Throughout the testing process, I used the following code for my `CustomLink` and `AirflowExtraLinkPlugin` in the `plugins/operator_link.py` file, and my DAG code.

### CustomLink and AirflowExtraLinkPlugin

```python
from airflow.sdk.bases.operatorlink import BaseOperatorLink
from airflow.plugins_manager import AirflowPlugin

class CustomLink(BaseOperatorLink):
name: str = "CustomLink"

def get_link(self, operator, *, ti_key) -> str:
return "https://www.google.com"

class AirflowExtraLinkPlugin(AirflowPlugin):
name = "airflow_extra_link_plugin"
operator_extra_links = [CustomLink(),]
```

### DAG Code

```python
from airflow.sdk import DAG
from datetime import datetime

from include.operators import CustomOperator

with DAG(
dag_id="try_extra_links",
start_date=datetime(2020, 1, 1),
schedule="@once"
) as dag:

custom_operator = CustomOperator(
task_id="custom_operator",
value=1
)

custom_operator__mapped = CustomOperator.partial(
task_id="custom_operator__mapped"
).expand(value=[1, 2, 3])

custom_operator >> custom_operator__mapped
```

### operator_extra_links set at the Class-level ✅

* In the `custom_operator` Task, I was able to see the button with `CustomLink`, which took me to `google.com`.
* In the `custom_operator__mapped`, I was also able to see the `CustomLink` button for each mapped Task instance.

Please see the Operator code for this here:

```python
from airflow.sdk.bases.operator import BaseOperator
from plugins.operator_links import CustomLink

class CustomOperator(BaseOperator):

operator_extra_links = (CustomLink(),)

def __init__(self, value, **kwargs):
super().__init__(**kwargs)
self.value = value

def execute(self, context):
print("Running custom operator to render operator_extra_links!")
```

### operator_extra_links set in the __init__ Method ❌

* In the `custom_operator` Task, I was able to see the button with `CustomLink`, which took me to `google.com`.
* In the `custom_operator__mapped`, I was NOT able to see the `CustomLink` button for any of the mapped Task instances.

Please see the Operator code for this here:

```python
from airflow.sdk.bases.operator.baseoperator import BaseOperator
from plugins.operator_links import CustomLink

class CustomOperator(BaseOperator):

def __init__(self, value, **kwargs):
super().__init__(**kwargs)
self.value = value
self.operator_extra_links = (CustomLink(), )

def execute(self, context):
print("Running custom operator to render operator_extra_links!")
```

### Operating System

MacOS

### Versions of Apache Airflow Providers

_No response_

### Deployment

Astronomer

### Deployment details

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the difference between the class-level and __init__ assignments in the DAG code, using plugins/operator_link.py and plugins/operator_links.py as shown. Trace BaseOperator and the dynamically-mapped task path, then add coverage for mapped instances; done means each mapped Task instance renders the CustomLink button in its Details tab.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.