Automated checks where possible for example dags
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Description
As described in https://cwiki.apache.org/confluence/display/AIRFLOW/Examples+Refurbish
For all checks that are possible to be automated (e.g. enforcing proper pydoc per example dag) checks are implemented.
This ticket might be splitted and does not define a fixed list but candidates might be:
- Check for pydoc on module as well as function/task level
- Check that display names are attached to all dags and tasks
- Check that doc_md is provided
- Check that tasks are properly typed (via mypy strict mode and ruff?)
- Check that tag "example" is added for all examples, tag according to provider and if testing then the tag "testing"
- No deprecated functions being used (via ruff?)
### Use case/motivation
Airflow Examples have been grown in number and focus over the past years. They purpose multiple things:
Serve as tutorials to learn Airflow DAG implementation
Serve with code snippets for documentation
Serve for testing the setup
(some) service for CI integration testing
Some example DAGs are in a good quality, some are not following best practices. Current examples do not follow a structure.
There are example DAGs contained in the Airflow core (currently pushed to standard provider/example_dags) as well as there are more examples in other providers. But examples from other providers are lot loaded automatically.
So in the [Airflow 3 Dev Calls](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+3+Dev+call%3A+Meeting+Notes#Airflow3Devcall:MeetingNotes-22May2025) there was a demand named to clean-up and optimize example DAGs.
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with the Examples Refurbish proposal and inspect the example DAGs in standard provider/example_dags. Identify which candidate checks are feasible and agree on a focused scope before implementing them. Done means the selected automated checks are enforced for the relevant example DAGs and pass against the existing examples.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, data-engineering, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100