apache / apache/airflow

CLI JSON output is polluted by logs and cannot be parsed by jq

Open
#66,139 7 comments 0 reactions 0 assignees View on GitHub
area:CLI kind:bug needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Under which category would you file this issue?

Airflow Core

### Apache Airflow version

3.2.0

### What happened and how to reproduce it?

## Issue Description

When running an Airflow CLI command with JSON output, logs are printed before the JSON payload.

For example:

```bash
airflow dags list -o json
prints Airflow/Alembic log lines before the JSON array starts.

This makes the output invalid JSON and breaks standard command-line tools such as jq.

## Steps to reproduce
1. Run:
airflow dags list -o json
2. Try to parse the output with jq:
airflow dags list -o json | jq
3. Or redirect the output to a file:
airflow dags list -o json > /tmp/output.json 2>&1
head -10 /tmp/output.json

## Observed behavior
The command outputs log lines before the JSON content, for example:
2026-04-30T01:14:29.770306Z [info] setup plugin alembic.autogenerate.schemas
2026-04-30T01:14:29.770491Z [info] setup plugin alembic.autogenerate.tables
2026-04-30T01:14:29.922347Z [warning] Could not import graphviz...
[{"dag_id": "SAFE_MODE_TEST_66104", ...}]
Because the output does not start with valid JSON, parsing fails:
jq: parse error: Invalid numeric literal at line 1, column 14
The failed pipe can also produce a Python BrokenPipeError traceback.

##Screenshots
##JSON parsing failure with jq

Image

##Raw JSON output polluted by logs

Image

##Redirected output file starts with logs instead of JSON

Image

### What you think should happen instead?

When using `-o json`, the command output should be valid JSON that can be parsed directly by tools such as `jq`.

Logs and warnings should not be written to the same output stream as the JSON payload. They should either be suppressed for machine-readable output or written to stderr so stdout remains valid JSON.

### Operating System

Ubuntu 24.04.4 LTS (WSL2)

### Deployment

Virtualenv installation

### Apache Airflow Provider(s)

standard

### Versions of Apache Airflow Providers

apache-airflow-providers-common-compat==1.14.3
apache-airflow-providers-common-io==1.7.2
apache-airflow-providers-common-sql==1.34.0
apache-airflow-providers-smtp==2.4.5
apache-airflow-providers-standard==1.12.3

### Official Helm Chart version

Not Applicable

### Kubernetes Version

_No response_

### Helm Chart configuration

_No response_

### Docker Image customizations

_No response_

### Anything else?

This occurs consistently in my local Airflow 3.2.0 environment.

The issue affects scripting and automation because `-o json` output is expected to be machine-readable.

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing `airflow dags list -o json` and piping it to `jq`, then trace how the Airflow CLI configures logging and writes the JSON response. Done means stdout contains only valid JSON that `jq` parses successfully, while Airflow/Alembic logs and warnings are suppressed or sent to stderr.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.