microsoft / microsoft/SPTAG

the SPTAGClient.AnnClient.Search method treats query and input vectors differently wrt normalization

Open
#82 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

customer raised need_triage
Dominant language
C++
Stars
5k
Forks
622
Avg merge
8h 37m
Merged PRs (30d)
5

Description

Bug description
when we build an SPTAGClient.AnnClient object off of a ANN server which loaded a Index.DistCalcMethod=Cosine (default) index, I expect that if I Search for any vector in that index (including non-unit vectors) that the nearest neighbor returned by Search should be that vector itself, and it should have distance=0. the current behavior is to actually return 1 - np.linalg.norm(non_unit_vector)

To Reproduce
Steps to reproduce the behavior:

  1. copy the Singlebox Python Wrapper example in the GettingStart.md file
  2. edit the last two lines to replace L2 with Cosine
  3. run this file

Expected behavior
for 10-element query vectors [0, 0, ..., 0], [2, 2, ..., 2], and [4, 4, ..., 4], the cosine distance of every vector in the input index (all [n, n, ..., n] for `0 < n < 100) should be exactly 0 (they have different magnitudes but the same direction)

Observed Behavior
the measured distance is not 1 - CosineSim(x, y), but instead 1 - |x| * CosineSim(x, y). for the three test query vectors, this means we see

  • all 0s: [0, 0, 0]
  • all 2s: [-5.324554920196533, -5.324554920196533, -5.324554920196533] (note: -5.324554920196533 = 1 - 1 - np.sqrt(10 * (2 ** 2)) = 1 - np.linalg.norm(q[1])
  • all 4s: [-11.649109840393066, -11.649109840393066, -11.649109840393066] (note: -11.649109840393066 = 1 - 1 - np.sqrt(10 * (4 ** 2)) = 1 - np.linalg.norm(q[2])

Desktop (please complete the following information):
using the current Dockerfile build

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Singlebox Python Wrapper example in docs/GettingStart.md and trace the SPTAGClient.AnnClient.Search path used after changing the index distance method to Cosine. Reproduce the behavior with the Dockerfile build and the three specified query vectors. Done means non-unit vectors with the same direction return cosine distance 0, including the nearest vector itself.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, docker, numpy, python
Domain
api, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.