huggingface / huggingface/dataset-viewer
Children jobs are not created after `JobManagerCrashedError`
Open
bug
P1
- Dominant language
- Python
- Stars
- 899
- Forks
- 130
- Avg merge
- 9h 26m
- Merged PRs (30d)
- 14
Description
This would explain why we have ~500 `PreviousStepStillProcessingError` entries, while they should be temporary.
https://github.com/huggingface/dataset-viewer/pull/2758#issuecomment-2090300183
Contributor guide
Research direction
Start with the issue's linked discussion in pull request 2758 and trace how JobManagerCrashedError relates to child-job creation. Confirm the behavior using the reported PreviousStepStillProcessingError entries; done means children are created after the crash and these errors no longer remain persistent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100