apache / apache/airflow

DAG processor treats any zip-format file (.jar, .pptx, .docx, .xlsx, etc.) as a potential DAG bundle, not just .zip

Open Beginner friendly
#71,125 4 comments 0 reactions 0 assignees View on GitHub
affected_version:3.3 area:core area:dag-bundles area:DAG-processing kind:bug
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Description

`find_dag_file_paths` (airflow-core/src/airflow/utils/file.py) decides whether to attempt DAG discovery on a file using:

```python
if path.is_file() and (path.suffix == ".py" or zipfile.is_zipfile(path)):
if might_contain_dag(file_path, safe_mode):
file_paths.append(file_path)
```

`zipfile.is_zipfile()` is a content sniff (checks for the PK zip magic bytes/central directory), not an extension check. There is no accompanying `path.suffix == ".zip"` condition. Since the ZIP format underlies many common file types beyond `.zip` itself — `.jar`, `.pptx`, `.docx`, `.xlsx`, `.apk`, `.epub`, `.odt`, `.whl`, etc. — any of these dropped into a DAGs folder (e.g. a build artifact, a supporting doc, a packaged dependency) will pass this check and get opened and scanned via `might_contain_dag`, purely because it happens to share the underlying zip container format with Airflow's own zipped-DAG-bundle feature.

This was reported previously in #45718 with a `.pptx` file, but that issue was closed as invalid because the specific symptom described there (garbled metric names) turned out to be an unrelated stat-sanitization bug, not this zip-detection design question. The zip-detection behavior itself was never actually addressed.

### Impact

At minimum this is wasted work (opening and scanning irrelevant files on every DAG processor cycle). Depending on `might_contain_dag`'s heuristic and the archive's contents, it can also produce confusing log noise, and in the originally reported case, appears able to feed corrupted/unexpected data further into DAG processing.

### What you think should happen instead

Gate the zip-bundle branch on the file extension in addition to (or instead of) the content sniff, e.g.:

```python
if path.is_file() and (path.suffix == ".py" or (path.suffix == ".zip" and zipfile.is_zipfile(path))):
```

This preserves the documented `.zip` DAG-bundle behavior while no longer opening arbitrary non-`.zip` files that merely share the same underlying container format. `.airflowignore` is a viable per-deployment workaround today (matching by extension), but it means every deployment that happens to keep e.g. `.jar` files anywhere under its DAGs folder has to know to add this rule proactively rather than it being a non-issue by default.

### How to reproduce

1. Place any non-`.zip` PK-zip-format file (a `.jar`, `.pptx`, `.docx`, etc.) anywhere under the DAGs folder.
2. Wait for the DAG processor to walk the directory.
3. Observe that the file is opened and passed through `might_contain_dag` (visible via DAG processor debug logs), the same as a `.zip` DAG bundle would be, despite not being one.

### Are you willing to submit a PR?

- [X] Yes

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct

---
Drafted-by: Claude Code (Sonnet 5); reviewed by @seanmuth before posting

Contributor guide

Open the contributing guide

Research direction

Start in airflow-core/src/airflow/utils/file.py at find_dag_file_paths and trace how might_contain_dag is reached. Reproduce with a non-.zip PK-format file and a .zip DAG bundle, then verify that only Python files and supported ZIP bundles are considered for DAG discovery, while unrelated archive-based formats are skipped.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.