apache / apache/airflow

All DAGs are being reserialised during each run of dag.test()

Open
#72,513 2 comments 0 reactions 0 assignees View on GitHub
kind:bug needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Under which category would you file this issue?

Airflow Core

### Apache Airflow version

3.3.1

### What happened and how to reproduce it?

1. Create several DAGs in a bundle.
2. Start debuggin one of them.
3. See in a terminal that the other one was reserialised.

VSCode's `launch.json`
```json
{
"version": "0.2.0",
"configurations": [
{
"name": "Current DAG",
"type": "debugpy",
"request": "launch",
"program": "${file}",
"console": "integratedTerminal",
"justMyCode": true,
]
}
```

### What you think should happen instead?

Debugging DAG and it's downstream dependencies should be reserialised only.

### Operating System

Debian GNU/Linux 13 (trixie)

### Deployment

Astronomer

### Apache Airflow Provider(s)

_No response_

### Versions of Apache Airflow Providers

apache-airflow-providers-celery==3.23.1
apache-airflow-providers-common-compat==1.18.0
apache-airflow-providers-common-io==1.8.0
apache-airflow-providers-common-sql==2.1.1
apache-airflow-providers-elasticsearch==6.9.0
apache-airflow-providers-fab==3.8.0
apache-airflow-providers-google==22.1.0
apache-airflow-providers-http==6.0.5
apache-airflow-providers-openlineage==2.20.1
apache-airflow-providers-postgres==7.0.2
apache-airflow-providers-redis==4.5.0
apache-airflow-providers-smtp==3.0.3
apache-airflow-providers-standard==1.18.0
apache-airflow-providers-tableau==5.6.1

### Official Helm Chart version

Not Applicable

### Kubernetes Version

_No response_

### Helm Chart configuration

_No response_

### Docker Image customizations

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue using the reported VSCode launch.json and the dag.test() entry point with several DAGs in one bundle. Trace which DAGs are serialized during debugging and compare that with the current DAG's downstream dependencies. Done means unrelated DAGs are not reserialized while the debugged DAG and its dependencies still are.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.