Project-MONAI / Project-MONAI/MONAILabel
Fundamental/possible "error"/feature
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 891
- Forks
- 269
- Avg merge
- 15h 41m
- Merged PRs (30d)
- 1
Description
I'm using the slicer/monailabel duo with CT images (82) and going on the single label route (several times).
As I observed the training process, I realised a possible "cheat"/"error" in the system which can be misleading: While the segmentation process is going on, I use auto-learning; every time I upload a new label to the virtual server, it restarts the learning process. The validation dice is usually great in the first 3-10 epochs.
As the learning restarts, it resets the training/validation pool. I believe the "last times" training gets into the "next times" validation pool. This is not a big problem but probably gives misleading accuracy measures.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the auto-learning restart triggered when a new label is uploaded, then inspect how the training and validation pools are reset between runs. Confirm whether samples from earlier training can enter later validation, and consider the issue done when validation data remains independent across restarts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100