binpash / binpash/pash

The sigpipe signals we issue cause early termination in distributed exec mode.

Open
#539 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Shell
Stars
601
Forks
52
PR merge metrics
No merged PRs in 30d

Description

When using distributed execution mode the subgraph scripts terminate early causing incorrect results. I was able to track the issue to the termination code used in eager.sh and dgsh_tee.sh. It seems like the sigpipe is being sent before the sockets finish writing. I commented out the code causing this in #537 .

It might also be useful to recheck the test cases that required this solution as there have been multiple fixes in autosplit and r_split recently that resolved issues related to files hanging and sigpipe signals.

Contributor guide

No contributing guide indexed for this repository

Research direction

Review the termination code in eager.sh and dgsh_tee.sh, along with the test cases that originally required it; issue #537 contains a related change. Reproduce distributed execution and verify that subgraph scripts no longer terminate before sockets finish writing, while the relevant autosplit and r_split cases still pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
shell
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.