opencv / opencv/opencv-python

Thin plate spline transform gives solid color for certain inputs

Open
#1,139 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
5.4k
Forks
1k
Avg merge
22h 17m
Merged PRs (30d)
3

Description

Certain combinations of points for a thin plate spline transformation cause the transformed image to be one solid color. Removing the offending combination of points will cause the image to transform as expected.

Below is code that reads the attached image and applys a transformation which should do nothing to the image. The first transformation erroneously turns the whole image a single color. The second transformation uses one less point and correctly does nothing to the image. Im not sure what makes combinations of points not compatible with each other. The points in the code are outside of the uploaded image, since the points were originally being used on a much larger image. I can't upload that here due to file size restrictions. However behavior on the smaller image is the same. Versions are Python 3.13.5, opencv-contrib-python 4.12.0.88, opencv-python 4.12.0.88, Windows 10.

import cv2
import numpy

# Variable initilization
img = cv2.imread('test.png')
base_points = numpy.array([[ 121, 5213],
                    [6197, 5197],
                    [1627, 3481],
                    [3146, 3477],
                    [4668, 3473],
                    [1616, 1752],
                    [4655, 1744],
                    [6179, 1742],
                    [  87,   29]])

# In this example a tps transformation is done.
# The input and output points are the same so the image shouldn't change.
# The whole image will be the color of the one of the corner pixels.
points = base_points[:]
points = points.reshape((-1,points.shape[0],2)).astype(numpy.int32)
tps = cv2.createThinPlateSplineShapeTransformer()
matches = [cv2.DMatch(i, i, 0) for i in range(points.shape[1])]
tps.estimateTransformation(points, points, matches)
warped1 = tps.warpImage(img)
cv2.imshow('Whole image is one color', warped1)

# This example uses one less point.
# The output is the input image, which is what would be expected.
points = base_points[1:]
points = points.reshape((-1,points.shape[0],2)).astype(numpy.int32)
tps = cv2.createThinPlateSplineShapeTransformer()
matches = [cv2.DMatch(i, i, 0) for i in range(points.shape[1])]
tps.estimateTransformation(points, points, matches)
warped2 = tps.warpImage(img)
cv2.imshow('Image is as expected', warped2)

cv2.waitKey(0)

Correct image: Image

Single color image: Image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided Python reproduction with test.png and the reported Python 3.13.5 and OpenCV 4.12.0.88 versions. Compare the nine-point and eight-point transformations, then determine where the failure belongs and verify that the nine-point identity transformation no longer produces a solid-color image.

Written by the indexing model from the issue text.

Assessment

Tech stack
opencv, python
Domain
computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.