apache / apache/airflow

Add Real-World DAG Debugging Strategies to Airflow Docs

Open
#71,948 5 comments 0 reactions 0 assignees View on GitHub
good first issue kind:documentation
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Description

Right now, most of the documentation around debugging Airflow DAGs references `dag.test()` and the command line. While this is a viable way to debug DAGs, it's not typically the way that it is done at the enterprise-level.

Update this docs page to include more "practical" or E2E testing techniques for DAGs. This might include running a Task and inspecting its XComs. Or re-running a single Task via the UI to validate that it ran successfully. There are tons of ways that this is done!

Existing Docs: https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/debug.html

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start with the existing debug documentation page at https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/debug.html and review its dag.test() and command-line guidance. Research practical enterprise DAG debugging approaches such as inspecting XComs and re-running a task from the UI, then update the page with useful E2E techniques and verify the documentation builds correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering, documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.