microsoft / microsoft/semantic-kernel

Python: Bug: RedisJsonCollection.delete() silently fails when prefix_collection_name_to_key_names is enabled

Open
#13,904 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug python
Dominant language
C#
Stars
28.6k
Forks
4.8k
Avg merge
14h 13m
Merged PRs (30d)
18

Description

Describe the bug

RedisJsonCollection._inner_delete does not apply the collection-name prefix to keys before calling JSON.DEL. When prefix_collection_name_to_key_names=True, the upsert path stores keys as {collection_name}:{key} but the delete path sends JSON.DEL {key} (without the prefix). The command targets a non-existent key, returns 0, and the record is never deleted.

Wire format captured via redis-cli MONITOR:

# Upsert (correct — prefixed):
JSON.SET "sk_cov_xxx:cov-1" "$" "{...}"

# Delete (wrong — not prefixed):
JSON.DEL "cov-1" "."

# Subsequent get still returns the record:
JSON.MGET "sk_cov_xxx:cov-1" "$"  →  [{"content": "alpha", ...}]

The hashset sibling RedisHashsetCollection._inner_delete correctly calls self._get_redis_key(key). The JSON version does not.

Expected behavior

collection.delete("cov-1") on a RedisJsonCollection with prefix_collection_name_to_key_names=True should send JSON.DEL "sk_cov_xxx:cov-1" and the record should be removed.

To Reproduce

Prerequisites: a Redis Stack server reachable on localhost:6379, redis-cli, and a working Python SK dev environment.

  1. Install Python SK per python/DEV_SETUP.md:

    cd python && make install-sk PYTHON_VERSION=3.12
    
  2. Start Redis Stack on port 6379:

    podman run -d -p 6379:6379 docker.io/redis/redis-stack:latest
    
  3. In a separate terminal, attach MONITOR:

    redis-cli -h localhost -p 6379 MONITOR
    
  4. From python/, run the following script:

    # repro_json_delete.py
    import asyncio
    from dataclasses import dataclass, field
    from typing import Annotated
    from uuid import uuid4
    
    from semantic_kernel.connectors.redis import RedisJsonCollection
    from semantic_kernel.data.vector import VectorStoreField, vectorstoremodel
    
    
    @vectorstoremodel
    @dataclass
    class MyModel:
        vector: Annotated[
            list[float] | None,
            VectorStoreField("vector", index_kind="hnsw", dimensions=3,
                             distance_function="cosine_similarity", type="float"),
        ] = None
        id: Annotated[str, VectorStoreField("key", type="str")] = field(
            default_factory=lambda: str(uuid4())
        )
        content: Annotated[str, VectorStoreField("data", type="str")] = "hello"
    
    
    async def main():
        async with RedisJsonCollection(
            record_type=MyModel,
            collection_name="repro_delete",
            prefix_collection_name_to_key_names=True,
        ) as col:
            await col.ensure_collection_deleted()
            await col.ensure_collection_exists()
    
            rec = MyModel(id="test-1", content="alpha", vector=[0.1, 0.2, 0.3])
            await col.upsert([rec])
    
            # Verify record exists
            fetched = await col.get("test-1")
            print("before delete:", fetched)  # MyModel(...)
    
            # Delete — this silently fails
            await col.delete("test-1")
    
            # Record is still there
            fetched = await col.get("test-1")
            print("after delete:", fetched)  # MyModel(...) — should be None
    
            await col.ensure_collection_deleted()
    
    
    asyncio.run(main())
    
    REDIS_CONNECTION_STRING="redis://localhost:6379" uv run python repro_json_delete.py
    

    Output:

    before delete: MyModel(vector=[0.1, 0.2, 0.3], id='test-1', content='alpha')
    after delete: MyModel(vector=[0.1, 0.2, 0.3], id='test-1', content='alpha')
    
  5. Check MONITOR output — the JSON.DEL targets the raw key test-1 instead of repro_delete:test-1.

Root cause

RedisJsonCollection._inner_delete (line 708 on main) passes raw keys directly:

async def _inner_delete(self, keys: Sequence[str], **kwargs: Any) -> None:
    await asyncio.gather(*[self.redis_database.json().delete(key, **kwargs) for key in keys])

Compare to RedisHashsetCollection._inner_delete (line 580) which correctly prefixes:

async def _inner_delete(self, keys: Sequence[TKey], **kwargs: Any) -> None:
    await self.redis_database.delete(*[self._get_redis_key(key) for key in keys])

Platform

  • Language: Python
  • Source: main branch of microsoft/semantic-kernel (SK version 1.41.2)
  • redis-py version: 6.4.0
  • Backend tested: Redis Stack 7.4.7 (RediSearch v21020, ReJSON v20809)
  • IDE: Kiro
  • OS: macOS 25.4.0 (Darwin), arm64

Additional context

The bug is silent — no error is raised. JSON.DEL on a non-existent key returns 0, which redis-py does not treat as an error. The caller has no indication the delete failed.

The bug only manifests when prefix_collection_name_to_key_names=True. The default for RedisJsonCollection is False, which is why the existing integration tests (which use the default) do not catch it. However, the parent class RedisCollection defaults to True, and any user who explicitly enables prefixing (a common pattern for multi-collection deployments) will hit this.

Test coverage gap

Tried to use the Redis connector and ran into issues — vector search didn't work and deletes with prefix_collection_name_to_key_names=True were silently failing. The existing integration tests (test_vector_store.py) only cover single-record upsert → get → delete with the default prefix setting (False) and never call collection.search(), so these paths have had zero test coverage. Added a new test file (test_redis_coverage.py) with 30 parametrised tests covering the full public surface — vector search, batch CRUD, filters, paging, include_vectors, prefix mode, etc. — and that's how these issues were found. The new tests should be included alongside the fix to prevent regression.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at RedisJsonCollection._inner_delete, identified around line 708, and compare it with RedisHashsetCollection._inner_delete around line 580. Reproduce the issue with Redis Stack using the provided script, then add coverage for prefix_collection_name_to_key_names=True alongside the existing test_vector_store.py coverage. Done means deleting a prefixed record removes it and the regression test passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, redis
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.