eduardolundgren / eduardolundgren/tracking.js

Finding too little descriptors and matches are wrong

Open
#246 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
9.5k
Forks
1.4k
PR merge metrics
No merged PRs in 30d

Description

I'm using the corner detection from here: https://github.com/wellflat/imageprocessing-labs/tree/master/cv/corner_detection

I transform the corners to be in the same format the descriptors function expects just like your example, yet I'm getting too little matches with `reciproalMatch()` and more with just `match()`, but both are pretty wrong.

Just the corners of the two images (overlaying on top of one another):
![image](https://user-images.githubusercontent.com/21977571/29748361-1a669872-8b1e-11e7-9393-91ae67458792.png)

no overlay:
![image](https://user-images.githubusercontent.com/21977571/29748365-30f8dcbc-8b1e-11e7-9f60-5f5aafe67784.png)

reciprocalMatch:
![image](https://user-images.githubusercontent.com/21977571/29748356-f325dd9a-8b1d-11e7-9d12-ac7eeaeb7bdb.png)

match:
![image](https://user-images.githubusercontent.com/21977571/29748352-eddaa884-8b1d-11e7-866a-9157c6dc7013.png)

Code:
```
function descriptors(imageData1, coordCorners1, imageData2, coordCorners2){
window.descriptorLength = 256;
window.matchesShown = 30;
tracking.Brief.N = window.descriptorLength;

const corners1 = flattenCornersArray(coordCorners1);
const corners2 = flattenCornersArray(coordCorners2);

var gray1 = tracking.Image.grayscale(imageData1.data, width, height);
var gray2 = tracking.Image.grayscale(imageData2.data, width, height);

// var corners1 = tracking.Fast.findCorners(gray1, width, height);
// var corners2 = tracking.Fast.findCorners(gray2, width, height);

var descriptors1 = tracking.Brief.getDescriptors(gray1, width, corners1);
var descriptors2 = tracking.Brief.getDescriptors(gray2, width, corners2);

var matches = tracking.Brief.reciprocalMatch(corners1, descriptors1, corners2, descriptors2);

matches.sort(function (a, b) {
return b.confidence - a.confidence;
});

const canvas = $('#canvas')[0];
const context = canvas.getContext('2d');
context.lineWidth = 3;
for (var i = 0; i < Math.min(window.matchesShown, matches.length); i++) {
var color = '#ffff00';
context.fillStyle = color;
context.strokeStyle = color;
context.fillRect(matches[i].keypoint1[0], matches[i].keypoint1[1], 4, 4);
context.fillRect(matches[i].keypoint2[0] , matches[i].keypoint2[1], 4, 4);

context.beginPath();
context.moveTo(matches[i].keypoint1[0], matches[i].keypoint1[1]);
context.lineTo(matches[i].keypoint2[0] , matches[i].keypoint2[1]);
context.stroke();
}
}

function flattenCornersArray(corners){
const arr = [];
for(let c of corners){
arr.push(c.coord.x, c.coord.y);
}
return arr;
}
```

Why doesn't it match properly?

PS: Adding blur doesn't help.

Orginal images:

![image](https://user-images.githubusercontent.com/21977571/29748377-917cc33c-8b1e-11e7-921a-526413935049.png)

![image](https://user-images.githubusercontent.com/21977571/29748378-97f9c944-8b1e-11e7-96ba-66bc71111b7d.png)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the issue's descriptors() reproduction and the linked cv/corner_detection example; inspect tracking.Brief.getDescriptors and reciprocalMatch with the flattened corner arrays. Reproduce the mismatch on the supplied images, then identify and correct the cause so the reported correspondences are valid and the matching result is verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.