No logs by journald if alloc is in pending state
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
### Nomad version
`Nomad v1.2.6`
### Operating system and Environment details
```bash
❯ lsb_release -a
LSB Version: n/a
Distributor ID: EndeavourOS
Description: EndeavourOS Linux
Release: rolling
Codename: rolling
```
### Issue
I have configured `journald` as the logging driver for Docker task plugin with the following config:
```hcl
config {
extra_labels = ["job_name", "job_id", "task_group_name", "task_name", "namespace", "node_name", "node_id"]
logging {
type = "journald"
config {
tag = "nomad"
labels = "com.hashicorp.nomad.alloc_id,com.hashicorp.nomad.job_id,com.hashicorp.nomad.job_name,com.hashicorp.nomad.namespace,com.hashicorp.nomad.node_id,com.hashicorp.nomad.node_name,com.hashicorp.nomad.task_group_name,com.hashicorp.nomad.task_name"
}
}
}
```
The issue is, if the `alloc` is in an unhealthy state where the application fails to start because of a runtime error, `nomad alloc logs` or logs on the UI are completely blank. It makes it really hard to debug why the application is failing to start when there are no logs to troubleshoot.
### Reproduction steps
1. Here's a simple `nginx` deployment Job file:
```hcl
job "nginx" {
datacenters = ["dc1"]
group "nginx" {
count = 1
network {
port "http" {
static = 8080
}
}
service {
name = "nginx"
port = "http"
}
task "nginx" {
driver = "docker"
config {
image = "nginx"
ports = ["http"]
volumes = [
"local:/etc/nginx/conf.d",
]
}
template {
data = <
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the issue with the nginx job, the Docker journald logging configuration, and the invalid nginx configuration. Compare the container output in journalctl with nomad alloc logs while the allocation is Pending. Done means startup error logs remain accessible through Nomad’s allocation logs and UI when journald is configured.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker
- Domain
- devops, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100