deezer / deezer/spleeter

[Discussion] reproduce evaluation table

Open
#512 1 comment 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
28.4k
Forks
3.1k
PR merge metrics
No merged PRs in 30d

Description

I can't reproduce SDR value with model trained with only Musdb data
I trained model with commend lines below

spleeter train --verbose -p configs/musdb_config.json -d
spleeter evaluate -p configs/musdb_config.json -o musdb_trained_spleeter_results --mus_dir -m

and I got the result like below
![스크린샷 2020-11-06 오후 4 12 49](https://user-images.githubusercontent.com/56575087/98336811-ebc29a00-204a-11eb-8fbd-d8a143956808.png)
and this page(https://github.com/deezer/spleeter/wiki/Separation-Performances) suggests that it should be larger then I got.
Did I miss something?

+) this is my training result
INFO:tensorflow:Saving dict for global step 200000: absolute_difference = 1.523042, bass_spectrogram = 0.19801405, drums_spectrogram = 0.35646605, global_step = 200000, loss = 1.523042, other_spectrogram = 0.56257313, vocals_spectrogram = 0.4059888
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 200000: musdb_model/model.ckpt-200000
INFO:tensorflow:Loss for final step: 1.1176548.
INFO:spleeter:Model training done

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the commands shown with configs/musdb_config.json and the referenced MUSDB path, then compare the evaluation output with the Separation Performances wiki page. Review the attached evaluation result and the training log ending at global step 200000; done means identifying why the reproduced SDR value differs or documenting the missing training or evaluation condition.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
audio-video-rtc, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.