apache / apache/airflow

[CLI command only] Importing `Variable` from `airflow.sdk` causes an `ImportError: cannot import name 'SUPERVISOR_COMMS'` for top-level Variable when running `airflow dags reserialize`

Open
#48,554 14 comments 1 reaction 0 assignees View on GitHub
affected_version:3.0 area:core kind:bug priority:low
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

EDIT: this only affects the CLI command `airflow dags reserialize`, the dag-processor handles the dag fine and it runs correctly.

### Apache Airflow version

3.0.0

### If "Other Airflow 2 version" selected, which one?

main

### What happened?

This dag serializes and runs fine for `from airflow.models import Variable` but not for `from airflow.sdk import Variable`

```
from airflow.sdk import dag, Variable
from airflow.decorators import task
# from airflow.models import Variable

# This one only works for imports from airflow.models, not from airflow.sdk
var1 = Variable.get("v1", "TOP_LEVEL_VAR")

@dag()
def test_var():

@task
def get_var():
print("TOP LEVEL VAR ", var1)

# this one works for both imports, from airflow.sdk and from airflow.models
var2 = Variable.get("v2", "TASK_LEVEL_VAR")
print("TASK LEVEL VAR ", var2)

get_var()

test_var()
```

The error is only caused when the Variable is used at the top level of the dag, not when within a task.

Error:

```
Traceback (most recent call last):
File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/variable.py", line 53, in get
return _get_variable(key, deserialize_json=deserialize_json)
File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/context.py", line 185, in _get_variable
from airflow.sdk.execution_time.task_runner import SUPERVISOR_COMMS
ImportError: cannot import name 'SUPERVISOR_COMMS' from 'airflow.sdk.execution_time.task_runner' (/opt/airflow/task-sdk/src/airflow/sdk/execution_time/task_runner.py)
```

### What you think should happen instead?

The dag should serialize and run with the updated import.

### How to reproduce

Add the above dag to your env, run airflow dags reserialize, see the error. Switch the import of Variable to `airflow.models` to see the dag work.

### Operating System

MacOS

### Versions of Apache Airflow Providers

None

### Deployment

Other

### Deployment details

breeze

### Anything else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with `airflow dags reserialize` using the example DAG and the `airflow.sdk` import. Start with `airflow/sdk/definitions/variable.py`, `airflow/sdk/execution_time/context.py`, and `airflow/sdk/execution_time/task_runner.py`; done means top-level `Variable.get` no longer raises the import error and the DAG serializes successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.