Project-MONAI / Project-MONAI/MONAI

Affine transformation order in AffineGrid wrong?

Open
#6,062 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

community Contribution wanted need discussions
Dominant language
Python
Stars
8.7k
Forks
1.6k
Avg merge
5d 1h
Merged PRs (30d)
20

Description

Describe the bug
I thought that the transformation order when applying affine transformations matters since the transformations can effect each other.

I thought the correct order is:

  • Scale
  • Rotate
  • Translate

The order in AffineGrid is:

  • Scale
  • Translate
  • Shear
  • Rotate
affine = torch.eye(spatial_dims + 1, device=_device)
if self.rotate_params:
    affine = affine @ create_rotate(spatial_dims, self.rotate_params, device=_device, backend=_b)
if self.shear_params:
    affine = affine @ create_shear(spatial_dims, self.shear_params, device=_device, backend=_b)
if self.translate_params:
    affine = affine @ create_translate(spatial_dims, self.translate_params, device=_device, backend=_b)
if self.scale_params:
    affine = affine @ create_scale(spatial_dims, self.scale_params, device=_device, backend=_b)

Scaling affects the size of the object, which in turn affects the position and orientation of the object. Therefore, scaling should be applied first.
Rotation affects the orientation of the object, but not its position. Therefore, it should be applied after scaling.
Translation affects the position of the object, but not its size or orientation. Therefore, it should be applied last.
(not quite sure what to do with the order of shear and rotate)

Shouldn't this then be:

affine = torch.eye(spatial_dims + 1, device=_device)
if self.translate_params:
    affine = affine @ create_translate(spatial_dims, self.translate_params, device=_device, backend=_b)
if self.rotate_params:
    affine = affine @ create_rotate(spatial_dims, self.rotate_params, device=_device, backend=_b)
if self.shear_params:
    affine = affine @ create_shear(spatial_dims, self.shear_params, device=_device, backend=_b)
if self.scale_params:
    affine = affine @ create_scale(spatial_dims, self.scale_params, device=_device, backend=_b)

To Reproduce
N/A

Expected behavior
Correct order of affine transformations.

Screenshots
N/A

Environment
N/A

Additional context
N/A

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the AffineGrid matrix construction and the create_rotate, create_shear, create_translate, and create_scale helpers shown in the issue. Check their matrix convention and how transformation order is defined, then verify the intended behavior with the project’s existing coverage. Done means the order is established and the implementation or documentation matches that decision.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.