[BUG] Why Training on musdb16hq halts or does not complete
- Dominant language
- Python
- Stars
- 28.4k
- Forks
- 3.1k
- PR merge metrics
- No merged PRs in 30d
Description
- [x] I didn't find a similar issue already open.
- [x] I read the documentation (README AND Wiki)
- [x] I have installed FFMpeg
- [x] My problem is related to Spleeter only, not a derivative product (such as Webapplication, or GUI provided by others)
## Description
## Step to reproduce
1. Installed using `.pip install python..`
2. Run as `administrator...`
3. Got `.NO..` error
## Output
```bash
After I run ...spleeter train -p 4stems-finetune.json -d E:\musdb18hq
I never get the training complete msg.
```
## Environment
| | |
| ----------------- | ------------------------------- |
| OS | Windows10|
| Installation type | pip |
| RAM available | 16Gb|
| Hardware spec | GTX 760 GPU / CPU Intel(R) Core(TM) i5 CPU 750 @ 2.67GHz 2.66 GHz|
## Additional context
I ran training on musdb_config.json and 4stems-finetune.json many times repeatedly and its starts as expected then gets about midway or 3/4 of the ways then just hangs for hours doing nothing. I also tried using only half of the musdb dataset and did the same thing. After 1 week of trying to complete this task, with multiple downloads of the musdb dataset, it still keeps failing. All my datasets contains 100 train / 50 test files. I ran the evaluation on the incompleted training results and my table is way off, I'm not sure what's going on and why its halting midway or 3/4 of the way. Am I missing files?..or corrupt files?..incompatabilities?...I'm running python 3.8.10 and spleeter 2.3.0
Any help or suggestions would be appreciated.
Contributor guide
Research direction
Start by reproducing `spleeter train -p 4stems-finetune.json -d E:\musdb18hq` with the reported Python 3.8.10, Spleeter 2.3.0, and Windows 10 environment. Compare the behavior with `musdb_config.json`, the reduced dataset, and the stated hardware; done means identifying a reproducible cause for the mid-training hang or documenting the missing information needed to isolate it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100