Project-MONAI / Project-MONAI/tutorials
maisi unconditioned inference return noise
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 2.5k
- Forks
- 803
- Avg merge
- 6d 22h
- Merged PRs (30d)
- 3
Description
Hallo,
I tried longer to make the diffusion unet (rectified flow version) work but without success. the VAE performes correct and encoding and reconstruction look fine. I take all the configs from the repo so configuration as well as weights are equivalently loaded as in https://github.com/Project-MONAI/tutorials/blob/main/generation/maisi/maisi_inference_tutorial.ipynb.
Using the diffusion unet with the existing code of diffusion_model_infer.py with
from scripts.diff_model_infer import run_inference
data = run_inference(
args,
device,
autoencoder,
diffusion_unet,
1.4,#scale_factor,
None,#top_region_index_tensor,
None, #bottom_region_index_tensor,
torch.tensor([[1.5,1.5,4.0]]).to('cuda'),#spacing_tensor,
torch.tensor([1]).to('cuda'),#modality_tensor,
(96,96,96),#output_size,
4,
logger,
)
I saw that there was an issue with the architecture which was different during training and the implementation but I'm not sure if that is related.
https://github.com/Project-MONAI/MONAI/issues/7991.
Some observations I made is when I would like to just reconstruction an image and basically make one step, the recon get's extremly noise.
For any suggestion what could be the potential issue, I'm thankful.
Best,
Lukas
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with scripts/diff_model_infer.py and the MAISI inference tutorial notebook, reproducing the one-step reconstruction that produces noise. Compare the inference architecture and loaded configuration with the training implementation, including the differences discussed in issue 7991. Done means the unconditioned inference or reconstruction no longer returns noisy output under the supplied settings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100