apache / apache/airflow

Python debugger stucks when the first python_callable from a class has **context parameter

Open
#51,861 6 comments 2 reactions 0 assignees View on GitHub
area:core kind:bug needs-triage priority:medium
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Apache Airflow version

3.0.2

### If "Other Airflow 2 version" selected, which one?

_No response_

### What happened?

Python debugger in VSCode starts infinite variable loading when the first `python_callable` handled by `PythonOperator` has the `**context` parameter. The problem was absent in Airflow 2.10.5.

P.S. Provided DAG is intentionally simplified. The problem heavily affects our workflow in a production environment.

**Case 1:**
- 1st callable doesn't have the `**context` parameter.
- 2nd callable has the `**context` parameter.
- All works fine.

![Image](https://github.com/user-attachments/assets/64dc15e2-c4a1-401f-af15-ac6e0de38b50)

**Case 2:**
- 1st callable has the `**context` parameter.
- Debugger starts infinite variable loading and reluctantly responds to the Stop command.

![Image](https://github.com/user-attachments/assets/3d5c5daa-6813-4826-9e12-235e0326a0e4)

### What you think should happen instead?

Python debugger should show variables and works properly.

### How to reproduce

VSCode's `launch.json`:
```JSON
{
"version": "0.2.0",
"configurations": [
{
"name": "Current DAG",
"type": "debugpy",
"request": "launch",
"program": "${file}",
"console": "integratedTerminal",
"justMyCode": true
},
]
}
```

`test.py`
```Python
from airflow import DAG
from airflow.providers.standard.operators.python import PythonOperator

class TestClass:

def func_with_context(self, **context) -> None:
return None

def func_without_context(self) -> None:
return None

with DAG("test_dag") as dag:

run_func_with_context = PythonOperator(
task_id="run_func_with_context",
python_callable=TestClass().func_with_context
)

run_func_without_context = PythonOperator(
task_id="run_func_without_context",
python_callable=TestClass().func_without_context
)

# Leave only 1 line uncommented at a time
# run_func_with_context >> run_func_without_context
run_func_without_context >> run_func_with_context

if __name__ == "__main__":
dag.test()

```

### Operating System

Debian GNU/Linux 12 (bookworm)

### Versions of Apache Airflow Providers

apache-airflow-providers-celery==3.11.0
apache-airflow-providers-common-compat==1.7.0
apache-airflow-providers-common-io==1.6.0
apache-airflow-providers-common-sql==1.27.1
apache-airflow-providers-fab==2.2.0
apache-airflow-providers-google==15.1.0
apache-airflow-providers-http==5.3.0
apache-airflow-providers-postgres==6.2.0
apache-airflow-providers-redis==4.1.0
apache-airflow-providers-smtp==2.1.0
apache-airflow-providers-standard==1.2.0

### Deployment

Other Docker-based deployment

### Deployment details

Extended image based on `apache/airflow:slim-3.0.2-python3.12`

### Anything else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start with the provided test.py reproducer, its PythonOperator callables, dag.test(), and the launch.json debugpy configuration. Run both dependency orders in VSCode and trace the first callable's context handling; done means variables load normally and the debugger responds to Stop in either order.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, vscode
Domain
backend, devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.