CLI JSON output is polluted by logs and cannot be parsed by jq
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 484
Description
### Under which category would you file this issue?
Airflow Core
### Apache Airflow version
3.2.0
### What happened and how to reproduce it?
## Issue Description
When running an Airflow CLI command with JSON output, logs are printed before the JSON payload.
For example:
```bash
airflow dags list -o json
prints Airflow/Alembic log lines before the JSON array starts.
This makes the output invalid JSON and breaks standard command-line tools such as jq.
## Steps to reproduce
1. Run:
airflow dags list -o json
2. Try to parse the output with jq:
airflow dags list -o json | jq
3. Or redirect the output to a file:
airflow dags list -o json > /tmp/output.json 2>&1
head -10 /tmp/output.json
## Observed behavior
The command outputs log lines before the JSON content, for example:
2026-04-30T01:14:29.770306Z [info] setup plugin alembic.autogenerate.schemas
2026-04-30T01:14:29.770491Z [info] setup plugin alembic.autogenerate.tables
2026-04-30T01:14:29.922347Z [warning] Could not import graphviz...
[{"dag_id": "SAFE_MODE_TEST_66104", ...}]
Because the output does not start with valid JSON, parsing fails:
jq: parse error: Invalid numeric literal at line 1, column 14
The failed pipe can also produce a Python BrokenPipeError traceback.
##Screenshots
##JSON parsing failure with jq
##Raw JSON output polluted by logs
##Redirected output file starts with logs instead of JSON
### What you think should happen instead?
When using `-o json`, the command output should be valid JSON that can be parsed directly by tools such as `jq`.
Logs and warnings should not be written to the same output stream as the JSON payload. They should either be suppressed for machine-readable output or written to stderr so stdout remains valid JSON.
### Operating System
Ubuntu 24.04.4 LTS (WSL2)
### Deployment
Virtualenv installation
### Apache Airflow Provider(s)
standard
### Versions of Apache Airflow Providers
apache-airflow-providers-common-compat==1.14.3
apache-airflow-providers-common-io==1.7.2
apache-airflow-providers-common-sql==1.34.0
apache-airflow-providers-smtp==2.4.5
apache-airflow-providers-standard==1.12.3
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### Anything else?
This occurs consistently in my local Airflow 3.2.0 environment.
The issue affects scripting and automation because `-o json` output is expected to be machine-readable.
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start by reproducing `airflow dags list -o json` and piping it to `jq`, then trace how the Airflow CLI configures logging and writes the JSON response. Done means stdout contains only valid JSON that `jq` parses successfully, while Airflow/Alembic logs and warnings are suppressed or sent to stderr.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100