mapillary / mapillary/OpenSfM

root_feature generates nan value with zero division

Open
#1,014 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.8k
Forks
899
PR merge metrics
No merged PRs in 30d

Description

I encountered an error that happened during Flann feature matching. The reason is one of the feature descriptors for one specific image has all nan values. After further investigation, the problem is caused by the raw feature descriptor being all zeros, based on how root_feature is calculated, there will be a zero division and cause nan value.
https://github.com/mapillary/OpenSfM/blob/c798da16206aa52395eaef09ea4d7746028479d3/opensfm/features.py#L282-L288

This happens rarely and mostly probably due to a feature point having all 0 intensity pixels around it, here is an example dumped in the middle of the feature extraction (the problematic feature point is at the under-exposed dark area at the left side). It can be reproduced by reading the raw gray image and using the below function call to reproduce

from opensfm import features
import cv2
image = cv2.imread("path_to_image", 0) # read gray image 
features.extract_features_sift(image, {'sift_edge_threshold':10.0, 'sift_peak_threshold':0.066, 'feature_root':True}, 16000)

sample

I guess the solution is adding an epsilon value or checking if s is zero. But I'm not sure if keeping all-zero features is reasonable, maybe discarding them is better.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in opensfm/features.py at the linked lines where root_feature normalizes the descriptor, then run the provided extract_features_sift reproduction with the zero-valued image region. Decide how zero descriptors should be handled and verify that the reproduction no longer produces NaN feature values.

Written by the indexing model from the issue text.

Assessment

Tech stack
opencv, python
Domain
computer-vision
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.