nomad alloc logs -f not stopping after exit 0 ?
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
### Question
clarification: what is expected behavior for `nomad alloc -f`
1) should `nomad alloc -f $ALLOC` close after task terminated ?
2) or should the 'stream' stay open indefinitely, in which case, perhaps from a 'user' perspective
if 1) please handle this as bug report.
if 2) no need to read further ... although when there is, example, exit 1, the streaming stops, so it seems inconsistent, but then I guess I will just work with `timeout $NR nomad alloc -f` to avoid long runtimes in some cases.
### Nomad version
Output from `nomad version`
```
Nomad v1.10.4
BuildDate 2025-08-12T20:48:32Z
Revision 62b195aaa535b2159d215eaf89e6f4a455d6f686
Nomad v1.9.7
BuildDate 2025-03-11T09:07:15Z
Revision f8695974efdbaa9f80ace06f0eec5bd4d2501035+CHANGES
```
### Operating system and Environment details
`Ubuntu 22.04.5`
### Issue
`nomad alloc logs -f $ALLOC` seems to stay 'alive' tailing logs, even if the allocation terminated with exit 0.
### Reproduction steps
```
cat << EOF > nomad-hello-world.hcl
job "helloworld" {
datacenters = ["*"]
type = "batch"
group "example" {
task "alpine" {
driver = "docker"
config {
image = "alpine"
args = [
"echo",
"hello_world",
]
}
}
}
}
EOF
nomad job plan nomad-hello-world.hcl
nomad job run -check-index 0 nomad-hello-world.hcl
export MAX_TRIES=20 TRY=0 ALLOC_ID=""
#loop until alloc done downloading image.
while [[ -z "$ALLOC_ID" && $TRY -lt $MAX_TRIES ]]; do
ALLOC_ID=$(nomad job status helloworld | awk '/run.*(running|complete).*ago/ {getline; print $1}')
if [[ -z "$ALLOC_ID" ]]; then
sleep 2
((TRY++))
echo "Waiting... ($TRY/$MAX_TRIES)"
fi
done
#show alloc status -short
nomad alloc status -short $ALLOC_ID
#echo hello_world should be quick, timeout with 60 seconds.
timeout 60 nomad alloc logs -f $ALLOC_ID
ALLOC_EXITCODE=$?
if test $ALLOC_EXITCODE -eq 0
then
echo "ALLOC_EXITCODE:$ALLOC_EXITCODE = OK -> PASS .. logs -f terminated properly"
else
nomad alloc status $ALLOC_ID
echo "ERROR ALLOC_EXITCODE=$ALLOC_EXITCODE timeout reached, logs -f was not closing after task dead/terminated/exit (0)"
fi
```
#### Expected Result
to get `ALLOC_EXITCODE:0 = OK -> PASS .. logs -f terminated properly`
#### Actual Result
`ERROR ALLOC_EXITCODE=124 timeout reached, logs -f was not closing after task dead/terminated/exit (0)`
```
+ Job: "helloworld"
+ Task Group: "example" (1 create)
+ Task: "alpine" (forces create)
Scheduler dry-run:
- All tasks successfully allocated.
Job Modify Index: 0
To submit the job with version verification run:
nomad job run -check-index 0 nomad-hello-world.hcl
When running the job with the check-index flag, the job will only be run if the
job modify index given matches the server-side version. If the index has
changed, another user has modified the job and the plan's results are
potentially invalid.
==> View this job in the Web UI: https://nomad-stagingmyinternal-domain.com/ui/jobs/helloworld@default
==> 2025-09-04T09:25:59+08:00: Monitoring evaluation "97ec7bc6"
2025-09-04T09:25:59+08:00: Evaluation triggered by job "helloworld"
2025-09-04T09:26:00+08:00: Allocation "5c78ab54" created: node "03e2e1fc", group "example"
2025-09-04T09:26:00+08:00: Evaluation status changed: "pending" -> "complete"
==> 2025-09-04T09:26:00+08:00: Evaluation "97ec7bc6" finished with status "complete"
==> View job details and metrics in the Web UI: https://nomad-stagingmyinternal-domain.com/ui/jobs/helloworld@default
Error querying allocation: Unexpected response code: 500 (alloc lookup failed: index error: Invalid UUID: encoding/hex: invalid byte: U+0068 'h')
Waiting... (1/20)
==> View job details and metrics in the Web UI: https://nomad-staging.myinternal-domain.com/ui/jobs/helloworld@default
ID = 5c78ab54-40d2-8191-3e3e-b5d71c736719
Name = helloworld.example[0]
Created = 4s ago
Modified = 2s ago
Tasks
Name State Last Event Time Lifecycle
alpine dead Terminated 2025-09-04T09:26:00+08:00 main
==> View allocation details in the Web UI: https://nomad-staging.myinternal-domain.com/ui/allocations/5c78ab54-40d2-8191-3e3e-b5d71c736719
hello_world
ID = 5c78ab54-40d2-8191-3e3e-b5d71c736719
Eval ID = 97ec7bc6
Name = helloworld.example[0]
Node ID = 03e2e1fc
Node Name = nomad-staging-node1
Job ID = helloworld
Job Version = 0
Client Status = complete
Client Description = All tasks have completed
Desired Status = run
Desired Description =
Created = 1m4s ago
Modified = 1m2s ago
Task "alpine" is "dead"
Task Resources:
CPU Memory Disk Addresses
100 MHz 300 MiB 300 MiB
Task Events:
Started At = 2025-09-04T01:26:00Z
Finished At = 2025-09-04T01:26:00Z
Total Restarts = 0
Last Restart = N/A
Recent Events:
Time Type Description
2025-09-04T09:26:00+08:00 Terminated Exit Code: 0
2025-09-04T09:26:00+08:00 Started Task started by client
2025-09-04T09:25:59+08:00 Driver Downloading image
2025-09-04T09:25:59+08:00 Task Setup Building Task Directory
2025-09-04T09:25:59+08:00 Received Task received by client
==> View allocation details in the Web UI: https://nomad-staging.myinternal-domain.com/ui/allocations/5c78ab54-40d2-8191-3e3e-b5d71c736719
ERROR ALLOC_EXITCODE=124 timeout reached, logs -f was not closing after task dead/terminated/exit (0)
```
### Job file (if appropriate)
```
job "helloworld" {
datacenters = ["*"]
type = "batch"
group "example" {
task "alpine" {
driver = "docker"
config {
image = "alpine"
args = [
"echo",
"hello_world",
]
}
}
}
}
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the `nomad alloc logs -f` command and reproduce the provided Alpine batch job on Ubuntu using the supplied shell script. Compare behavior for exit code 0 with the reported exit-1 case, then verify that the command exits successfully after the completed allocation while still streaming logs as intended.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bash, docker, go
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 62/100