[Bug] Exact Euclidean search violates translation invariance with large offsets (Top-1 changes)
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 34.7k
- Forks
- 2.7k
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 187
Description
Current Behavior
When all vectors and the query are translated by the same large offset (e.g. +1_000_000), the exact Top-k ordering changes even though Euclidean distance is mathematically invariant under translation.
Actual output:
base: [(192, 0.9025439), (152, 0.94426656), (214, 1.0027068), (219, 1.4205707), (212, 3.6855476)]
translated: [(152, 0.89486384), (192, 0.92280143), (214, 0.9762812), (219, 1.4045128), (212, 3.6625214)]
base ids: [192, 152, 214, 219, 212]
translated ids: [152, 192, 214, 219, 212]
BUG REPRODUCED
Top-1 changes from point 192 to point 152.
Steps to Reproduce
- Start a clean Qdrant container:
docker run -p 6333:6333 qdrant/qdrant:latest - Save the following script as
reproduce.py:
import argparse
import os
from qdrant_client import QdrantClient, models
BASE_COLLECTION = "mr_translation_l2_base"
TRANSLATED_COLLECTION = "mr_translation_l2_trans"
OFFSET = 1_000_000.0
QUERY = [278.9473894406408, -141.9580317361888]
POINTS = [
(192, [279.7386493949869, -141.52387313280337]),
(152, [279.3407783212691, -141.09962429397626]),
(214, [278.1977839418044, -141.2920619385261]),
(219, [279.69353304090316, -140.74920807425514]),
(212, [275.9759470860143, -139.77771582489316]),
]
def translated(vector):
return [x + OFFSET for x in vector]
def disable_proxy_for_localhost():
for key in ("HTTP_PROXY", "HTTPS_PROXY", "ALL_PROXY", "http_proxy", "https_proxy", "all_proxy"):
os.environ.pop(key, None)
os.environ["NO_PROXY"] = "localhost,127.0.0.1"
def recreate_collection(client, name):
if client.collection_exists(name):
client.delete_collection(name)
client.create_collection(
collection_name=name,
vectors_config=models.VectorParams(
size=2,
distance=models.Distance.EUCLID,
),
hnsw_config=models.HnswConfigDiff(m=0, full_scan_threshold=10),
optimizers_config=models.OptimizersConfigDiff(indexing_threshold=0),
)
def load_points(client):
recreate_collection(client, BASE_COLLECTION)
recreate_collection(client, TRANSLATED_COLLECTION)
client.upsert(
BASE_COLLECTION,
[models.PointStruct(id=point_id, vector=vector) for point_id, vector in POINTS],
wait=True,
)
client.upsert(
TRANSLATED_COLLECTION,
[models.PointStruct(id=point_id, vector=translated(vector)) for point_id, vector in POINTS],
wait=True,
)
def search(client, collection_name, query):
response = client.query_points(
collection_name,
query=query,
limit=5,
search_params=models.SearchParams(exact=True),
with_payload=False,
with_vectors=False,
)
return [(point.id, point.score) for point in response.points]
def cleanup(client):
for name in (BASE_COLLECTION, TRANSLATED_COLLECTION):
if client.collection_exists(name):
client.delete_collection(name)
def main():
parser = argparse.ArgumentParser(description="Reproduce Qdrant exact Euclidean search translation-invariance violation.")
parser.add_argument("--url", default="http://localhost:6333", help="Qdrant HTTP URL.")
parser.add_argument("--timeout", type=float, default=60.0, help="Client timeout in seconds.")
parser.add_argument("--keep-collections", action="store_true", help="Do not delete reproduction collections after running.")
args = parser.parse_args()
disable_proxy_for_localhost()
client = QdrantClient(url=args.url, timeout=args.timeout)
try:
print("server info:", client.info())
load_points(client)
base = search(client, BASE_COLLECTION, QUERY)
trans = search(client, TRANSLATED_COLLECTION, translated(QUERY))
base_ids = [point_id for point_id, _ in base]
trans_ids = [point_id for point_id, _ in trans]
print("base:", base)
print("translated:", trans)
print("base ids:", base_ids)
print("translated ids:", trans_ids)
if base_ids != trans_ids:
print("BUG REPRODUCED")
raise SystemExit(1)
print("PASS")
finally:
if not args.keep_collections:
cleanup(client)
if __name__ == "__main__":
main()
3.Run: python reproduce.py
4.Observe the output: Top-1 id changes from 192 → 152 under exact=True.
Expected Behavior
The Top-k ordering (and especially the Top-1 point) must be identical before and after translation, because:
$|(q + c) - (r + c)|_2 = |q - r|_2$
for any constant (c).
Possible Solution
In exact search path, compute raw Euclidean score using f64 (higher precision) instead of f32, or center vectors before scoring to avoid catastrophic cancellation.
Context (Environment)
Qdrant server: 1.17.1
Docker image: qdrant/qdrant:latest
Python client: qdrant-client 1.17.1
No HNSW, no quantization, no filters, exact=True only.
This breaks the guarantee of "exact" search when vectors have large absolute values.
Detailed Description
Exact brute-force Euclidean search is not invariant under uniform translation of all vectors + query. The root cause is in the f32-based scoring:
lib/segment/src/spaces/simple.rs:214(euclid_similarity)lib/segment/src/vector_storage/query_scorer/metric_query_scorer.rs- When values reach ~1e6, f32 loses enough precision that
(a - b).powi(2)produces wrong ordering.
Possible Implementation
- Add
f64path for exact L2 scoring inMetricQueryScorer. - Or automatically center vectors (subtract mean) before exact scoring.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Run the provided reproduce.py script against Qdrant 1.17.1 to confirm the exact-search ordering change. Read lib/segment/src/spaces/simple.rs at euclid_similarity and lib/segment/src/vector_storage/query_scorer/metric_query_scorer.rs to trace the f32 scoring path. Done means exact Euclidean results preserve the same Top-k ordering after uniformly translating the vectors and query, including the supplied large-offset case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python, rust
- Domain
- backend-api-design, databases, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100