Project-MONAI / Project-MONAI/MONAI

multiprocessing metatensor efficiency?

Open
#6,472 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Contribution wanted enhancement
Dominant language
Python
Stars
8.7k
Forks
1.6k
Avg merge
5d 1h
Merged PRs (30d)
20

Description

Describe the bug

follow up of #6468, may require benchmarking https://github.com/Project-MONAI/MONAI/blob/c2a9a31beb22e1d3321016be7e68e0875cf2a8ad/monai/data/__init__.py#L121-L124

map_classes_to_indices utility (and ClassesToIndices transforms)
currently returns a list of coordinates for each class, but the type is
List[ of MetaTensors], where each class coordinate sub-list is its own
MetaTensor. This PR changes it to return a list of torch.Tensors (or
ndarray), since we don't need a MetaTensor here.

I ran into an issue with current MetaTensor list, where it would just
freeze without any errors, when trying to save the cached indices
(returned from ClassesToIndices to cache, which is ListProxy shared
mem). It randomly happens, but much more frequently when the number of
classes is large (e.g. 105 output classes, so ClassesToIndices returns a
list of 105 MetaTensors). I'm not sure what the cause of the freeze is,
but my guess is that ListProxy tries to pickle each element of this list
(and struggles with MetaTensors). Disabling MetaTensor return type here,
solves the issue.

cc @myron

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Begin with monai/data/init.py lines 121-124 and trace map_classes_to_indices and ClassesToIndices, focusing on how their returned values are cached through multiprocessing. Benchmark the current and proposed return types, especially with many classes, and verify that saving cached indices no longer freezes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
data, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.