apache / apache/airflow

dag parse stdout goes to warning on celery worker

Open
#41,982 0 comments 0 reactions 0 assignees View on GitHub
area:logging kind:meta provider:celery
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Body

not sure why but here's an example

sample code

```
from time import sleep

from airflow import DAG
from airflow.decorators import task
from airflow.models.baseoperator import chain_linear

with DAG(dag_id="dag1", catchup=False):

@task
def the_task():
sleep(10)

tasks = []
for num in range(3):
tasks.append(the_task.override(task_id=f"task_{num+1}")())

chain_linear(*tasks)

print("HI I AM HERE")
```

celery worker output

image

this can pollute worker logs if you have a really chatty dag

### Committer

- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Contributor guide

Open the contributing guide

Research direction

No file or test is named. Start by reproducing the sample DAG with a Celery worker and trace how stdout from DAG parsing is classified; done means ordinary DAG stdout no longer appears as a warning or pollutes worker logs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.