Action runner ignores SIGPIPE and causes shell script with redirections to hang
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 6.3k
- Forks
- 1.4k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 24
Description
Describe the bug
When running a script (https://github.com/btrfs/linux/actions/runs/5490218255/workflow) which runs fstests on a self-hosted VM, one of the tests will hang. This test is doing something like this
_fail() {
echo $1
exit 1
}
od /dev/urandom | dd of=/some/file bs=1M count=10 > somefile.txt 2>&1 || _fail "dd failed"
The boxes executing this line get stuck here, if you look at the /proc/pid/fd listing you can see
[root@xfstests6 fstests]# ll /proc/33527/fd
total 0
lr-x------ 1 root root 64 Jul 7 21:13 0 -> 'pipe:[51951]'
l-wx------ 1 root root 64 Jul 7 21:13 1 -> /root/fstests/results/btrfs_block_group_tree/btrfs/016.full
l-wx------ 1 root root 64 Jul 7 21:13 10 -> /tmp/30763.out
lr-x------ 1 root root 64 Jul 7 21:13 106 -> 'pipe:[53318]'
l-wx------ 1 root root 64 Jul 7 21:13 11 -> /tmp/30763.out
l-wx------ 1 root root 64 Jul 7 21:13 112 -> 'pipe:[53319]'
l-wx------ 1 root root 64 Jul 7 21:13 2 -> /root/fstests/results/btrfs_block_group_tree/btrfs/016.full
lr-x------ 1 root root 64 Jul 7 21:13 255 -> /root/fstests/tests/btrfs/016
and the ps auxw show
root 30742 5.7 2.6 3707952 102100 ? Sl 21:12 0:01 /root/actions-runner/bin/Runner.Worker spawnclient 106 112
so we're using those pipe's to capture the output of the job.
If I change this command to not do the redirects everything works fine.
Also I had a version of this runner that ssh'ed into the vm from a different machine to run this command, and this worked fine. It only breaks when the runner is on the machine that is trying to run this testsuite.
To Reproduce
Steps to reproduce the behavior:
- Start a local runner with fstests setup for btrfs (I can provide a series of steps to set this up if needed.)
- Create a workflow that runs ./check btrfs/016
- The task will hang
Expected behavior
The task should run just fine
Runner Version and Platform
actions-runner-linux-x64-2.305.0
OS of the machine running the runner? OSX/Windows/Linux/...
Fedora 38
What's not working?
The workflow is hanging when the runner is local to the machine running the testsuite.
Job Log Output
You can see my attempt here https://github.com/btrfs/linux/actions/runs/5490218255.
Runner and Worker's Diagnostic Logs
The logs don't show anything helpful, but I can provide them if needed.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Runner.Worker spawnclient entry point and reproduce the Fedora 38 workflow using the shown od|dd command with output redirections. Inspect how the local runner captures these pipes and handles SIGPIPE. Done means the fstests workflow completes instead of hanging, while the redirected command still reports failures through the shown _fail path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, shell
- Domain
- ci-cd, devops, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100