galaxyproject / galaxyproject/pulsar

Node failure job with no `return_code` file still marked as successful in Galaxy

Open
#358 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
46
Forks
62
Avg merge
3d 9h
Merged PRs (30d)
15

Description

Running hifiasm on the VGP instance and the node it executed on appears to have failed and the job was terminated prematurely. As a result, nothing past a certain point in the job script was executed, including capturing/writing the tool return code to `return_code`. Despite this, none of the postprocessing failures were fatal (due to being work_dir outputs) and for some reason, Galaxy still treated this job as successful.

Contributor guide

Open the contributing guide

Research direction

Trace how Pulsar handles a terminated job when the `return_code` file is absent, including whether failures in `work_dir` outputs affect the final status. Reproduce the hifiasm scenario if possible and verify that a job whose execution stops before writing `return_code` is reported as failed rather than successful.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.