apache / apache/airflow

GET /api/v2/dags returns empty list after dag-processor restart despite CLI showing DAGs correctly

Open
#69,249 2 comments 0 reactions 0 assignees View on GitHub
area:API area:core kind:bug needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Under which category would you file this issue?

Airflow Core

### Apache Airflow version

main (3.4.0), reproduced via Breeze dev environment

### What happened and how to reproduce it?

**Issue Description**

After restarting the `dag-processor` process mid-session (e.g. via Breeze's mprocs dashboard, or a container restart), `GET /api/v2/dags` returns an empty list:

{"dags":[],"total_entries":0}

despite the database containing 203 correctly-parsed DAGs, and `airflow dags list` (CLI) showing all of them without issue.

**Root cause (traced)**

- All 203 rows in the `dag` table have `is_stale = 1`, even though `last_parsed_time` is recent and matches the actual last dag-processor parse.
- The public `GET /api/v2/dags` endpoint applies `exclude_stale=True` by default (`airflow-core/src/airflow/api_fastapi/core_api/routes/public/dags.py`), which silently filters out every DAG.
- Passing `exclude_stale=false` explicitly returns the DAGs correctly.
- `airflow dags list` (CLI) does not apply the same `is_stale` filter, so it shows everything — creating a CLI vs API discrepancy.
- All rows in `dag_bundle` also have `version = NULL`.
- Running `airflow dags reserialize` immediately fixes it — `is_stale` flips to `0` for all 203 DAGs. This confirms `update_dags()` (`dag_processing/collection.py:587`, sets `dm.is_stale = False`) works correctly when invoked directly.
- However, the live, backgrounded `dag-processor` process, after being restarted, continues parsing DAG files (confirmed via updated `last_parsed_time`) but never clears `is_stale` on its own.
- Confirmed `deactivate_stale_dags()` (`dag_processing/manager.py:446`) is NOT the cause — no "Deactivating stale DAG" log lines appear, ruling out DAGs being re-marked stale after being cleared.

**Steps to reproduce**

1. `breeze start-airflow --python 3.10` (SQLite backend, default dev setup)
2. Let it fully start, confirm DAGs load in the UI/API normally
3. Restart the `dag-processor` process (e.g. via Breeze's mprocs dashboard, pressing `r` on `dag_processor`)
4. Wait for `dag-processor` to report UP again and complete at least one parse cycle
5. Query `GET /api/v2/dags` → returns `{"dags":[],"total_entries":0}`
6. Run `airflow dags list` via CLI → shows all DAGs correctly
7. Run `airflow dags reserialize` → `GET /api/v2/dags` now returns DAGs correctly

**Evidence**

-- Before reserialize
SELECT is_stale, COUNT(*) FROM dag GROUP BY is_stale;
-- (1, 203)

SELECT name, version FROM dag_bundle;
-- all 10 bundles, all version=None

-- After `airflow dags reserialize`
SELECT is_stale, COUNT(*) FROM dag GROUP BY is_stale;
-- (0, 203)

$ airflow dags list | wc -l
459

$ curl -s -H "Authorization: Bearer $TOKEN" http://localhost:8080/api/v2/dags
{"dags":[],"total_entries":0}

$ curl -s -H "Authorization: Bearer $TOKEN" "http://localhost:8080/api/v2/dags?exclude_stale=false&limit=5"
{"dags":[{"dag_id":"aggregate_regional_sales", ..., "is_stale":true, "bundle_version":null, ...}]}

**Relevant code**

- `airflow-core/src/airflow/api_fastapi/core_api/routes/public/dags.py` — `exclude_stale` filter applied by default
- `airflow-core/src/airflow/dag_processing/collection.py:587` — `update_dags()`, sets `dm.is_stale = False`
- `airflow-core/src/airflow/dag_processing/manager.py:446` — `deactivate_stale_dags()`, confirmed NOT the cause
- `airflow-core/src/airflow/models/dag.py:365` — `is_stale` column definition (`default=True`)

### What you think should happen instead?

`GET /api/v2/dags` should return the same set of DAGs that `airflow dags list` sees after a dag-processor restart, without requiring a manual `airflow dags reserialize`. Either the dag-processor's background parse loop should correctly clear `is_stale` after a restart, or the CLI and API should apply consistent staleness filtering.

### Operating System

Windows 11 + WSL2 (Ubuntu), Docker Desktop with WSL2 backend

### Deployment

Other

### Apache Airflow Provider(s)

_No response_

### Versions of Apache Airflow Providers

_No response_

### Official Helm Chart version

Not Applicable

### Kubernetes Version

Not Applicable

### Helm Chart configuration

Not Applicable

### Docker Image customizations

Not Applicable

### Anything else?

This occurs consistently (every time) after a dag-processor restart in my testing. Happy to provide full container logs if helpful — let me know what would be most useful.

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the restart sequence and querying GET /api/v2/dags, then read airflow-core/src/airflow/api_fastapi/core_api/routes/public/dags.py and the dag-processing paths in collection.py and manager.py. Compare is_stale and dag_bundle.version before and after reserialize; done means the API returns parsed DAGs after a dag-processor restart without requiring manual reserialize, with consistent CLI/API behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.