Project-MONAI / Project-MONAI/tutorials

MAISI missing labels for conditioning maisi-rflow with vista3D pseudo labels

Open
#2,047 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
2.5k
Forks
803
Avg merge
6d 22h
Merged PRs (30d)
3

Description

I’m training a pre-trained maisi-rflow model to generate new images conditioned on segmentation masks generated with vista3D.

In MAISI-data is written that the pseudo-labels can be produced using VISTA3D. I’m using MONAI bundle. Both MAISI and VISTA3D have labels IDs from 1 to 132 as can be compared in this two links:
1.https://github.com/Project-MONAI/tutorials/blob/main/generation/maisi/configs/label_dict.json
2. https://github.com/Project-MONAI/VISTA/blob/main/vista3d/data/jsons/label_dict.json
However, when I follow the MAISI inference tutorial and download the provided pseudo-labels, I see the label 200=body which does not exist in VISTA3D. As a result, when I condition maisi-rflow on my masks that only include IDs up to 132, generation quality degrades due to the missing label.

Which model or pipeline should I use to produce complete pseudo-labels compatible with maisi-rflow conditioning? Below are examples of generated images.

Image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Compare the linked MAISI and VISTA3D label_dict.json files and review the MAISI inference tutorial and its provided pseudo-labels. Trace where the 200=body label originates and determine which model or pipeline produces complete labels compatible with maisi-rflow conditioning. Done means the supported label mapping and generation path are documented or corrected.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.