deepset-ai / deepset-ai/haystack

InMemoryDocumentStore unique metadata count fails for list and dict values

Open
#12,746 1 comment 0 reactions 1 assignee View on GitHub

@sjrl is already working on this.

Since Sep 14, 2026.

Dominant language
Python
Stars
26.6k
Forks
3.2k
Avg merge
1d 3h
Merged PRs (30d)
194

Description

Describe the bug
InMemoryDocumentStore.count_unique_metadata_by_filter() raises TypeError when a requested metadata field contains a JSON-serializable list or dictionary. The async variant fails through the same implementation.

Error message

TypeError: unhashable type: list
TypeError: unhashable type: dict

Expected behavior
The method should count distinct JSON-serializable metadata values, including lists and dictionaries, consistently with the Document.meta contract. Equal list or dictionary values should count once, while list order should remain significant.

Additional context
Document.meta is documented as JSON-serializable. For the same documents, json.dumps(document.to_dict()), filter_documents(), and get_metadata_field_unique_values() work with these values. Existing scalar count tests pass, but composite metadata is not covered. The failure occurs when a user explicitly calls this public metadata statistics API; it is not part of the automatic retrieval path.

To Reproduce

from haystack import Document
from haystack.document_stores.in_memory import InMemoryDocumentStore

for metadata in ({"tags": ["news", "ai"]}, {"source": {"type": "docs", "section": {"id": 1}}}):
    store = InMemoryDocumentStore()
    store.write_documents([Document(content="Document", meta=metadata)])
    field = next(iter(metadata))
    store.count_unique_metadata_by_filter(filters={}, metadata_fields=[field])

Each call raises TypeError from the set-based unique-value collection. count_unique_metadata_by_filter_async() reproduces the same error.

FAQ Check

System:

  • OS: Linux
  • Haystack version: main at f7b46875f (2026-09-14)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.