Project-MONAI / Project-MONAI/MONAILabel
Decouple Image Signal and Guidance Signal
@danieltudosiu is already working on this.
Since Feb 3, 2023.
- Dominant language
- Python
- Stars
- 891
- Forks
- 269
- Avg merge
- 15h 41m
- Merged PRs (30d)
- 1
Description
Is your feature request related to a problem? Please describe.
At the moment there is no straightforward way to decouple the Image Signal and Guidance Signal from the DeepEdit (and I assume the rest of the library).
The problem stems from the usage of the base SupervisedTrainer and bypassing its limitations via a new iteration method that internally calls the default iteration function.
Describe the solution you'd like
I think DeepEdit should incorporate the Interaction logic into a new Trainer that allows the same flexibility with added ones such as specialised prepare_batch for both the internal loop as the external loop.
Furthermore, there is no default Key for the Signal similar to GanKeys or AdversarialKeys.
Lastly, currently, DeepEdit development is hindered by using the default prepare_batch that does not take into account the existence of Guidance Signal and its usage in specialised blocks such as:
- Bi-Directional Seed Attention Module.
- Hint Fusion Block/Interaction Guided Gating Network
- Inception-Atrous-Collation
- Cross-Attention in Transformer based networks (Swin U-NETR, U-NETR)
Additional context
Currently trying to improve the click ingestion of MONAI Label together with Self-Supervised Learning.
Edit: Added more examples of recent techniques requiring the decoupling of signals.
Edit 2: Added details about the lack of keys.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.