Less recall post-refinement

Describe the bug
when using refinement API, we found that the distances (for same pair of vectors) are different between ground truth and refinement, resulting in refinement not always capturing the k closest neighbors from the candidates.

See below for an example of the ground truth in the range of candidates but refinement failing to pick that out. 41363 is one of the candidates, and should be ground truth, but not getting identified by refinement(the line of output under "indices" is the result of refine ), selecting 9 correct candidates except 41363

Steps/Code to reproduce bug
Follow this guide http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports to craft a minimal bug report. This helps us reproduce the issue you're having and resolve the issue more quickly.

Expected behavior
Refine should be able to recognize the ground truth vector if it's in the candidates

Environment details (please complete the following information):

• Environment location: Bare-metal
• Method of RAFT install: conda

Additional context
Add any other context about the problem here.

Start by reproducing the refinement API case described in the issue using the linked minimal-bug-report guidance, comparing the ground-truth distances with refinement for the candidate containing vector 41363. Done means refinement selects the ground-truth vector whenever it is present among the candidates.