root_feature generates nan value with zero division
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.8k
- Forks
- 899
- PR merge metrics
- No merged PRs in 30d
Description
I encountered an error that happened during Flann feature matching. The reason is one of the feature descriptors for one specific image has all nan values. After further investigation, the problem is caused by the raw feature descriptor being all zeros, based on how root_feature is calculated, there will be a zero division and cause nan value.
https://github.com/mapillary/OpenSfM/blob/c798da16206aa52395eaef09ea4d7746028479d3/opensfm/features.py#L282-L288
This happens rarely and mostly probably due to a feature point having all 0 intensity pixels around it, here is an example dumped in the middle of the feature extraction (the problematic feature point is at the under-exposed dark area at the left side). It can be reproduced by reading the raw gray image and using the below function call to reproduce
from opensfm import features
import cv2
image = cv2.imread("path_to_image", 0) # read gray image
features.extract_features_sift(image, {'sift_edge_threshold':10.0, 'sift_peak_threshold':0.066, 'feature_root':True}, 16000)
I guess the solution is adding an epsilon value or checking if s is zero. But I'm not sure if keeping all-zero features is reasonable, maybe discarding them is better.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in opensfm/features.py at the linked lines where root_feature normalizes the descriptor, then run the provided extract_features_sift reproduction with the zero-valued image region. Decide how zero descriptors should be handled and verify that the reproduction no longer produces NaN feature values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- opencv, python
- Domain
- computer-vision
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100