facebookresearch / facebookresearch/silk

For the images captured with large camera rotation angles, the matching performance is much worse than that of SuperPoint.

Open
#69 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Gnuplot
Stars
679
Forks
66
PR merge metrics
No merged PRs in 30d

Description

I used the inference code,
```bash
CHECKPOINT=assets/models/silk/coco-rgb-aug.ckpt
TOPK=500
SIZE=640

./bin/silk-features -o -m pvgg-micro -c $CHECKPOINT -k $TOPK -d $OUT/features -s $SIZE $IMAGES/*.$IMAGES_EXTENSION
# generate image pairs to match
ls $OUT/features/*.pt | sort -V | xargs ./bin/generate-matches exhaustive -s > $OUT/matches.txt
# match keypoints from generated image pairs
./bin/silk-matching -o -m double-softmax -t 0.9 $OUT/matches.txt $OUT/matches/
# visualize matches
./bin/silk-viz image -o $OUT/viz $OUT/matches/*.pt
```

For images captured after a large rotation angle of the camera, the matching performance is much worse compared to SuperPoint.
For euroc dataset, matching result of silk model as follow:
![image](https://github.com/facebookresearch/silk/assets/3840711/2c349b05-9168-4553-b8c7-40192bbf4354)

matching result of superpoint model as follow:
![image](https://github.com/facebookresearch/silk/assets/3840711/2f2c147c-aa86-46a5-a57d-bc9772de9885)

Contributor guide

Open the contributing guide

Research direction

Reproduce the reported case with the provided silk-features, generate-matches, silk-matching, and silk-viz commands on images with large camera rotations, using the stated checkpoint and settings. Compare the resulting matching quality with the referenced SuperPoint result; done requires identifying the cause of the gap and demonstrating improved large-rotation matching.

Written by the indexing model from the issue text.

Assessment

Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.