Project-MONAI / Project-MONAI/MONAI

Spacingd transform possibly resampling incorrectly

Open
#7,529 7 comments 0 reactions 1 assignee View on GitHub

@atbenmurray is already working on this.

Since Mar 12, 2024.

Dominant language
Python
Stars
8.7k
Forks
1.6k
Avg merge
5d 1h
Merged PRs (30d)
20

Description

Describe the bug
I want to resample 3D microscopy data to be able to pass (almost) isotropic data through a 3D U-Net. I get data resampled along the wrong axes (to my understanding) and by the wrong factors.

To Reproduce
This highly simplified example shows the (to me) unexpected behavior:

# Parameters
in_pixdim = [1.0, 0.25, 0.25]
out_pixdim = [0.25, 0.25, 0.25]

# Create data
image = torch.zeros(in_tensor_size, dtype=torch.float32) 
image = MetaTensor(image, meta={'pixdim': in_pixdim})
label = torch.zeros(in_tensor_size, dtype=torch.int32)
label = MetaTensor(label, meta={'pixdim': in_pixdim})
data = {"image": image, "label": label}

# Check shapes and metadata (is space = 'RAS' correct for microscopy?)
image.shape, image.meta, label.shape, label.meta

(torch.Size([20, 100, 100]),
 {'pixdim': [1.0, 0.25, 0.25],
  affine: tensor([[1., 0., 0., 0.],
          [0., 1., 0., 0.],
          [0., 0., 1., 0.],
          [0., 0., 0., 1.]], dtype=torch.float64),
  space: RAS},
 torch.Size([20, 100, 100]),
 {'pixdim': [1.0, 0.25, 0.25],
  affine: tensor([[1., 0., 0., 0.],
          [0., 1., 0., 0.],
          [0., 0., 1., 0.],
          [0., 0., 0., 1.]], dtype=torch.float64),
  space: RAS})

# Set up transforms
sp_d = Spacingd(keys=["image", "label"], pixdim=out_pixdim, mode=("bilinear", "nearest"),)
sp = Spacing(pixdim=out_pixdim, mode="bilinear",)

# Apply transforms
sp_data = sp_d(data)
sp_image = sp(image)

# Check outputs
sp_data["image"].shape, sp_data["image"].meta, sp_data["label"].shape, sp_data["label"].meta

(torch.Size([20, **397, 397**]),
 {'pixdim': **[1.0, 0.25, 0.25]**,
  affine: tensor([[0.2500, 0.0000, 0.0000, 0.0000],
          [0.0000, 0.2500, 0.0000, 0.0000],
          [0.0000, 0.0000, 1.0000, 0.0000],
          [0.0000, 0.0000, 0.0000, 1.0000]], dtype=torch.float64),
  space: RAS},
 torch.Size([20, **397, 397**]),
 {'pixdim': **[1.0, 0.25, 0.25]**,
  affine: tensor([[0.2500, 0.0000, 0.0000, 0.0000],
          [0.0000, 0.2500, 0.0000, 0.0000],
          [0.0000, 0.0000, 1.0000, 0.0000],
          [0.0000, 0.0000, 0.0000, 1.0000]], dtype=torch.float64),
  space: RAS})

sp_image.shape, sp_image.meta

(torch.Size([20, **397, 397**]),
 {'pixdim': **[1.0, 0.25, 0.25]**,
  affine: tensor([[0.2500, 0.0000, 0.0000, 0.0000],
          [0.0000, 0.2500, 0.0000, 0.0000],
          [0.0000, 0.0000, 1.0000, 0.0000],
          [0.0000, 0.0000, 0.0000, 1.0000]], dtype=torch.float64),
  space: RAS})

# Try with another pair of in_pixdim and out_pixdim
in_pixdim = [1.0, 0.25, 0.25]
out_pixdim = [1.0, 0.25, 0.25]  # No change

# ... same code to transform the data ...

# Check outputs
sp_data["image"].shape, sp_data["image"].meta, sp_data["label"].shape, sp_data["label"].meta

(torch.Size([20, 100, **397**]),
 {'pixdim': **[1.0, 0.25, 0.25]**,
  affine: tensor([[1.0000, 0.0000, 0.0000, 0.0000],
          [0.0000, 0.2500, 0.0000, 0.0000],
          [0.0000, 0.0000, 1.0000, 0.0000],
          [0.0000, 0.0000, 0.0000, 1.0000]], dtype=torch.float64),
  space: RAS},
 torch.Size([20, 100, **397**]),
 {'pixdim': **[1.0, 0.25, 0.25]**,
  affine: tensor([[1.0000, 0.0000, 0.0000, 0.0000],
          [0.0000, 0.2500, 0.0000, 0.0000],
          [0.0000, 0.0000, 1.0000, 0.0000],
          [0.0000, 0.0000, 0.0000, 1.0000]], dtype=torch.float64),
  space: RAS})

sp_image.shape, sp_image.meta

(torch.Size([20, 100, **397**]),
 {'pixdim': **[1.0, 0.25, 0.25]**,
  affine: tensor([[1.0000, 0.0000, 0.0000, 0.0000],
          [0.0000, 0.2500, 0.0000, 0.0000],
          [0.0000, 0.0000, 1.0000, 0.0000],
          [0.0000, 0.0000, 0.0000, 1.0000]], dtype=torch.float64),
  space: RAS})

Expected behavior
In the first case, the 'pixdim' metadata value of the upscaled data should be [0.25, 0.25, 0.25] instead of [1.0, 0.25, 0.25] and the shape should be [80, 100, 100] instead of [20, 397, 397].

In the second case, the 'pixdim' metadata is correct (I guess this never gets changed), but the shape should be [20, 100, 100] (as the input) instead of [20, 100, 397]. Why did the x dimension get upscaled?

Environment

Please see below.

Ensuring you use the relevant python executable, please paste the output of:

python -c "import monai; monai.config.print_debug_info()"

================================
Printing MONAI config...

MONAI version: 1.3.0
Numpy version: 1.26.3
Pytorch version: 2.2.0+cu121
MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
MONAI rev id: 865972f7a791bf7b42efbcd87c8402bd865b329e
MONAI file: /home//miniconda3/envs/qute-env/lib/python3.11/site-packages/monai/init.py

Optional dependencies:
Pytorch Ignite version: NOT INSTALLED or UNKNOWN VERSION.
ITK version: NOT INSTALLED or UNKNOWN VERSION.
Nibabel version: 5.2.0
scikit-image version: 0.22.0
scipy version: 1.12.0
Pillow version: 10.2.0
Tensorboard version: 2.15.1
gdown version: NOT INSTALLED or UNKNOWN VERSION.
TorchVision version: 0.17.0+cu121
tqdm version: 4.66.1
lmdb version: NOT INSTALLED or UNKNOWN VERSION.
psutil version: 5.9.8
pandas version: 2.2.0
einops version: NOT INSTALLED or UNKNOWN VERSION.
transformers version: NOT INSTALLED or UNKNOWN VERSION.
mlflow version: NOT INSTALLED or UNKNOWN VERSION.
pynrrd version: 1.0.0
clearml version: NOT INSTALLED or UNKNOWN VERSION.

For details about installing the optional dependencies, please visit:
https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies

================================
Printing system config...

System: Linux
Linux version: Ubuntu 23.10
Platform: Linux-6.5.0-25-generic-x86_64-with-glibc2.38
Processor: x86_64
Machine: x86_64
Python version: 3.11.5
Process name: python
Command: ['python', '-c', 'import monai; monai.config.print_debug_info()']
Open files: []
Num physical CPUs: 4
Num logical CPUs: 8
Num usable CPUs: 8
CPU usage (%): [16.6, 11.4, 24.2, 45.3, 14.4, 14.0, 12.3, 64.2]
CPU freq. (MHz): 3075
Load avg. in last 1, 5, 15 mins (%): [11.7, 7.3, 7.3]
Disk usage (%): 81.8
Avg. sensor temp. (Celsius): UNKNOWN for given OS
Total physical memory (GB): 125.5
Available memory (GB): 105.9
Used memory (GB): 17.2

================================
Printing GPU config...

Num GPUs: 1
Has CUDA: True
CUDA version: 12.1
cuDNN enabled: True
NVIDIA_TF32_OVERRIDE: None
TORCH_ALLOW_TF32_CUBLAS_OVERRIDE: None
cuDNN version: 8902
Current device: 0
Library compiled for CUDA architectures: ['sm_50', 'sm_60', 'sm_70', 'sm_75', 'sm_80', 'sm_86', 'sm_90']
GPU 0 Name: Quadro RTX 5000
GPU 0 Is integrated: False
GPU 0 Is multi GPU board: False
GPU 0 Multi processor count: 48
GPU 0 Total memory (GB): 15.7
GPU 0 CUDA capability (maj.min): 7.5

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.