Help with projective transform

Hi!
I'm trying to implement a function that will calculate the projective transformation from a bunch of projected coordinates and their corresponding real world values. This is just another (equally relevant) way of creating a projective transformation (other than with the cam_rotation and cam_position).

The following has been inspired by matlab's version for the same task (nothing revolutionary):

function findProjectiveTransform(uv::Vector{T},xy::Vector{T}) where T <: AbstractArray
    M = length(xy)
    @assert length(uv) == M "same number of coordinates needed"
    @assert M > 3 "at least 4 coordinates required" 
    X = zeros(2M,8)
    for i = 1:M
        X[i,1:2] .= xy[i]
        X[i,3] = 1
        X[i,7:8] .= -first(uv[i])*xy[i]
    end
    for i = 1:M
        X[i+M,4:5] .= xy[i]
        X[i+M,6] = 1
        X[i+M,7:8] .= -last(uv[i])*xy[i]
    end
    U = vec(cat(2, uv...)')
    Tvec = X\U
    push!(Tvec, 1)
    return LinearMap(reshape(Tvec, 3, 3))
end

Can you guys help me understand what I'm doing wrong with that final step?
I think this could be a great addition to this package.

Start with the proposed findProjectiveTransform function and inspect how LinearMap and the existing cam_rotation/cam_position transformations are represented. Check the final matrix construction against the projected-coordinate examples, then define tests showing that the returned transformation maps corresponding real-world and projected coordinates correctly.