microsoft / microsoft/semantic-kernel

Python: Bug: Cosmos DB for MongoDB vector index uses the similarity code as the index kind

Open
#14,104 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug python triage
Dominant language
C#
Stars
28.6k
Forks
4.8k
Avg merge
14h 13m
Merged PRs (30d)
18

Description

Describe the bug
CosmosMongoCollection._get_index_definitions (python/semantic_kernel/connectors/azure_cosmos_db.py, line 401) sets
cosmosSearchOptions["kind"] from DISTANCE_FUNCTION_MAP_MONGODB — a similarity code ("COS"/"IP"/"L2") —
instead of INDEX_KIND_MAP_MONGODB (the index kind: "vector-ivf"/"vector-hnsw"/"vector-diskann").

This causes three problems:

  1. The createIndexes command sends an invalid kind — Cosmos DB for MongoDB vCore requires kind to be one of
    vector-ivf/vector-hnsw/vector-diskann, so vector-index creation fails against a live account.
  2. kind ends up equal to similarity.
  3. The match index_kind block (line 411) can never match a vector-* case, so the HNSW/IVF/DiskANN tuning options
    (m, efConstruction, numList, maxDegree, lBuild) are silently dropped.

The mapped value of INDEX_KIND_MAP_MONGODB is never read (it's only used for a membership check at line 392) — the
tell. The sibling NoSQL path does it correctly at line 149: "type": INDEX_KIND_MAP_NOSQL[field.index_kind].

To Reproduce
Deterministic unit-level repro (no live account needed):

import asyncio
from unittest.mock import AsyncMock, MagicMock
from pymongo import AsyncMongoClient
from semantic_kernel.connectors.azure_cosmos_db import CosmosMongoCollection
from semantic_kernel.data.vector import VectorStoreCollectionDefinition, VectorStoreField

definition = VectorStoreCollectionDefinition(fields=[
    VectorStoreField("key", name="id"),
    VectorStoreField("data", name="content"),
    VectorStoreField("vector", name="vector", dimensions=5,
                     index_kind="hnsw", distance_function="cosine_similarity"),
])
db = AsyncMock(); db.create_collection = AsyncMock(); db.command = AsyncMock()
client = AsyncMock(spec=AsyncMongoClient); client.get_database = MagicMock(return_value=db)
col = CosmosMongoCollection(collection_name="c", record_type=dict,
                            definition=definition, mongo_client=client, database_name="d")

asyncio.run(col.ensure_collection_exists(m=16, efConstruction=64))
opts = db.command.call_args.kwargs["command"]["indexes"][1]["cosmosSearchOptions"]
print(opts)
# Actual:   {'kind': 'COS', 'similarity': 'COS', 'dimensions': 5}
#           -> invalid kind, and m / efConstruction were silently dropped

Against a live Cosmos DB for MongoDB vCore account, ensure_collection_exists() fails because kind="COS" is not a valid vector index kind.

Expected behavior
cosmosSearchOptions["kind"] == "vector-hnsw" and cosmosSearchOptions["similarity"] == "COS" (the two must be distinct), and the HNSW tuning options (m, efConstruction) appear in cosmosSearchOptions.

Screenshots
N/A

Platform

  • Language: Python
  • Source: main branch of repository (also affects current pip releases)
  • AI model: N/A
  • IDE: VS Code
  • OS: Windows

Additional context
Root cause is a single wrong map lookup at line 401 (should be INDEX_KIND_MAP_MONGODB[field.index_kind]). I have a fix + tests ready and will open a PR shortly. I'll take this — PR incoming.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in python/semantic_kernel/connectors/azure_cosmos_db.py at CosmosMongoCollection._get_index_definitions, then compare the MongoDB path with the sibling NoSQL mapping at line 149. Run the deterministic AsyncMock reproduction and confirm the resulting options use a vector-* kind, retain the similarity code, and include the HNSW tuning values.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.