Project-MONAI / Project-MONAI/tutorials
Clarification on [1] in swinunetr_pretrained README and training pipeline of ssl_pretrained_weights.pth
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 2.5k
- Forks
- 803
- Avg merge
- 6d 22h
- Merged PRs (30d)
- 3
Description
Hi MONAI team,
I’m trying to understand the origin of the pretrained weight:
ssl_pretrained_weights.pth (from MONAI-extra-test-data, ~719MB).
From the checkpoint structure, it contains:
- encoder.*
- decoder*
- out.conv.*
- encoder.mask_token
which looks like a full encoder-decoder model.
I found the following description in the tutorial README:
https://github.com/Project-MONAI/tutorials/blob/main/self_supervised_pretraining/swinunetr_pretrained/README.md
"The entire SwinUNETR model including encoder and decoder was trained end-to-end using self-supervised learning techniques as outlined in [1]."
However, it is unclear what [1] refers to in terms of actual training code or pipeline.
Could you please help clarify:
-
What exactly does reference [1] correspond to?
(Is it a specific paper, or an internal training implementation?) -
Was
ssl_pretrained_weights.pthtrained using the
research-contributions/SwinUNETR/Pretrain(SSLHead: rotation + contrastive + reconstruction),
or a different encoder-decoder / autoencoder-style pretraining pipeline?
A short clarification would already help a lot.
Thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the self_supervised_pretraining/swinunetr_pretrained/README.md and its reference [1], then compare the documented pipeline with research-contributions/SwinUNETR/Pretrain and its SSLHead. Use the listed ssl_pretrained_weights.pth checkpoint keys to trace which pipeline produced it. Done when the README identifies [1] and clearly states the checkpoint's training pipeline.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pytorch
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100