spindleRunBE() failure prevents log file generation
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 110
- Forks
- 35
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 2
Description
A proposed `ci.yml` refactoring splits flux-ubuntu tests into four separate jobs and runs them in parallel on separate nodes. Three jobs cover SPINDLE_DEBUG=[1-3] and the fourth does not set SPINDLE_DEBUG.
The run in question succeeded for SPINDLE_DEBUG=[1,3] and the no-debugging option, but failed on SPINDLE_DEBUG=2 with the error message:
```
Run docker exec node-1 bash -c 'cd Spindle-build/testsuite && SPINDLE_DEBUG=2 flux alloc --nodes=${workers} ./runTests --nodes=${workers} --tasks-per-node=3'
docker exec node-1 bash -c 'cd Spindle-build/testsuite && SPINDLE_DEBUG=2 flux alloc --nodes=${workers} ./runTests --nodes=${workers} --tasks-per-node=3'
shell: /usr/bin/bash -e {0}
Running: ./run_driver --dependency --push
60.237s: flux-shell[0]: stderr: spindleRunBE failed!
Error: The action 'Run spindle-flux-ubuntu-debug2 testsuite' has timed out after 7 minutes.
```
At that point the CI script should extract the log files, but based on the `ls` the script performs, no files were generated. Looking at the likely code path, `LOGGING_FINI` is not called during any of the error paths, so the log files were likely not flushed to disk before spindle exited.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with ci.yml and the spindleRunBE error path referenced in the issue. Trace where LOGGING_FINI is called during failures, then run the SPINDLE_DEBUG=2 flux-ubuntu tests to verify that log files are flushed and available for CI extraction when the tests time out.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, docker
- Domain
- hpc, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100