apache / apache/airflow

Intermittent DAG Task Failures with Missing Logs ("No host supplied") using CeleryExecutor

Open
#56,388 3 comments 0 reactions 0 assignees View on GitHub
affected_version:3.1 area:core area:scheduler kind:bug needs-triage priority:high
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Apache Airflow version

3.1.0

### What happened?

### Deployment Details
- Executor: CeleryExecutor
- Installation method: Installed via pip (not Docker or Kubernetes)

### Issue Description
Tasks occasionally fail immediately at the first task without producing any logs. The Airflow UI shows the following error:
`Could not read served logs: Invalid URL 'http://:8793/log/dag_id=.../task_id=.../attempt=1.log': No host supplied`
After restarting Airflow services (webserver and scheduler), the problem temporarily resolves, but it reoccurs after days.

### Impact
This issue is intermittent but recurring, making it difficult to rely on Airflow for production workloads. Even when tasks appear to succeed after a service restart, the problem resurfaces after some days, causing sudden DAG failures with no logs

### Observations / Notes
- Restarting services temporarily fixes the issue.
- The problem seems related to hostname resolution or worker URL caching in the webserver.
- I monitored memory usage and noticed that the issue appears when system memory is nearly full — it seems that hostnames or related metadata are lost or not resolved correctly when memory pressure increases.
- The issue occurs intermittently and is hard to reproduce consistently, but becomes more likely over time as memory usage grows.

### What you think should happen instead?

The Airflow webserver should consistently resolve and serve valid log URLs for all tasks, regardless of uptime or restart state.
Tasks should not fail immediately due to missing or malformed log endpoints, and restarts should not be required to restore normal behavior.

### Operating System

Ubuntu 24.04.2 LTS

### Versions of Apache Airflow Providers

3.0.1

### Deployment

Official Apache Airflow Helm Chart

### Deployment details

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by investigating log serving between the webserver and CeleryExecutor workers, focusing on how worker hostnames and URLs are resolved after long uptime and under memory pressure. Reproduce the intermittent “No host supplied” error if possible; done means task logs remain accessible through valid URLs without requiring service restarts.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering, distributed-systems, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.