The sigpipe signals we issue cause early termination in distributed exec mode.
- Dominant language
- Shell
- Stars
- 601
- Forks
- 52
- PR merge metrics
- No merged PRs in 30d
Description
When using distributed execution mode the subgraph scripts terminate early causing incorrect results. I was able to track the issue to the termination code used in eager.sh and dgsh_tee.sh. It seems like the sigpipe is being sent before the sockets finish writing. I commented out the code causing this in #537 .
It might also be useful to recheck the test cases that required this solution as there have been multiple fixes in autosplit and r_split recently that resolved issues related to files hanging and sigpipe signals.
Contributor guide
No contributing guide indexed for this repository
Research direction
Review the termination code in eager.sh and dgsh_tee.sh, along with the test cases that originally required it; issue #537 contains a related change. Reproduce distributed execution and verify that subgraph scripts no longer terminate before sockets finish writing, while the relevant autosplit and r_split cases still pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- shell
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100