DAG processor treats any zip-format file (.jar, .pptx, .docx, .xlsx, etc.) as a potential DAG bundle, not just .zip
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
### Description
`find_dag_file_paths` (airflow-core/src/airflow/utils/file.py) decides whether to attempt DAG discovery on a file using:
```python
if path.is_file() and (path.suffix == ".py" or zipfile.is_zipfile(path)):
if might_contain_dag(file_path, safe_mode):
file_paths.append(file_path)
```
`zipfile.is_zipfile()` is a content sniff (checks for the PK zip magic bytes/central directory), not an extension check. There is no accompanying `path.suffix == ".zip"` condition. Since the ZIP format underlies many common file types beyond `.zip` itself — `.jar`, `.pptx`, `.docx`, `.xlsx`, `.apk`, `.epub`, `.odt`, `.whl`, etc. — any of these dropped into a DAGs folder (e.g. a build artifact, a supporting doc, a packaged dependency) will pass this check and get opened and scanned via `might_contain_dag`, purely because it happens to share the underlying zip container format with Airflow's own zipped-DAG-bundle feature.
This was reported previously in #45718 with a `.pptx` file, but that issue was closed as invalid because the specific symptom described there (garbled metric names) turned out to be an unrelated stat-sanitization bug, not this zip-detection design question. The zip-detection behavior itself was never actually addressed.
### Impact
At minimum this is wasted work (opening and scanning irrelevant files on every DAG processor cycle). Depending on `might_contain_dag`'s heuristic and the archive's contents, it can also produce confusing log noise, and in the originally reported case, appears able to feed corrupted/unexpected data further into DAG processing.
### What you think should happen instead
Gate the zip-bundle branch on the file extension in addition to (or instead of) the content sniff, e.g.:
```python
if path.is_file() and (path.suffix == ".py" or (path.suffix == ".zip" and zipfile.is_zipfile(path))):
```
This preserves the documented `.zip` DAG-bundle behavior while no longer opening arbitrary non-`.zip` files that merely share the same underlying container format. `.airflowignore` is a viable per-deployment workaround today (matching by extension), but it means every deployment that happens to keep e.g. `.jar` files anywhere under its DAGs folder has to know to add this rule proactively rather than it being a non-issue by default.
### How to reproduce
1. Place any non-`.zip` PK-zip-format file (a `.jar`, `.pptx`, `.docx`, etc.) anywhere under the DAGs folder.
2. Wait for the DAG processor to walk the directory.
3. Observe that the file is opened and passed through `might_contain_dag` (visible via DAG processor debug logs), the same as a `.zip` DAG bundle would be, despite not being one.
### Are you willing to submit a PR?
- [X] Yes
### Code of Conduct
- [X] I agree to follow this project's Code of Conduct
---
Drafted-by: Claude Code (Sonnet 5); reviewed by @seanmuth before posting
Contributor guide
Research direction
Start in airflow-core/src/airflow/utils/file.py at find_dag_file_paths and trace how might_contain_dag is reached. Reproduce with a non-.zip PK-format file and a .zip DAG bundle, then verify that only Python files and supported ZIP bundles are considered for DAG discovery, while unrelated archive-based formats are skipped.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100