simonsobs / simonsobs/sotodlib
Improve preprocessing error catching
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 19
- Forks
- 23
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 14
Description
There are some failure cases in the preprocessing functions that are not caught within those functions and therefore cause them to fail instead of returning error information, potentially with generic and less informative error message when being run from within a parallelized loop with MPI or multiprocessing. These include calls to get_meta in most of the processing or loading functions and failures within cleanup_mandb related to locked h5 files (see #1376).
These need to be wrapped in try excepts that return useful error outputs.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the preprocessing and loading functions that call get_meta, along with cleanup_mandb and its handling of locked H5 files described in #1376. Review how these functions report failures during MPI or multiprocessing, then run the relevant existing checks if available. Done means failures are caught within the functions and returned as useful error information instead of generic loop errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100