crewAIInc / crewAIInc/crewAI

[FEATURE] Add batch query recall (recall_many) to UnifiedMemory with single-call embedding

Open
#7,530 1 comment 0 reactions 1 assignee View on GitHub

@Rohitkanithi is already working on this.

Since Sep 17, 2026.

feature-request
Dominant language
Python
Stars
58.8k
Forks
8.5k
Avg merge
1d 15h
Merged PRs (30d)
109

Description

Feature Area

Performance optimization

Is your feature request related to a an existing bug? Please link it here.

NA

Describe the solution you'd like

Add a batch recall method Memory.recall_many(queries: list[str], ...) to crewai.memory.UnifiedMemory:

  • Single-Call Batch Embedding: Instead of sequential HTTP requests for each query, embed all query strings simultaneously in a single API call using the embedder's batch API (e.g. _encode_batch(queries)). Every major provider (OpenAI, Gemini, Cohere, Voyage AI, Ollama) natively accepts a list of strings in one request.
  • Concurrent Vector Search: Query the vector storage backend for each embedding in parallel.
  • Unified Cross-Query Deduplication & Ranking: Combine candidate matches across queries, deduplicate by record.id, compute composite scores (semantic, recency, importance), and return the top-ranked list[MemoryMatch].
  • Update RecallMemoryTool: Update RecallMemoryTool._run in crewai/tools/memory_tools.py to use self.memory.recall_many(queries) instead of looping sequentially.
Describe alternatives you've considered

Currently, RecallMemoryTool accepts a list of queries (queries: list[str]), but executes them in a sequential loop:

for query in queries:
    matches = self.memory.recall(query, limit=20)  # Sequential network round-trip each time

For N queries, this incurs N consecutive round-trips to the embedding API. With typical API latency of 250-400ms per request, searching 4 queries takes 1.2-1.6 seconds, during which the agent is blocked waiting for I/O.

Additional context
  • UnifiedMemory already has batch embedding on the write path (_encode_batch() used by remember_many()). Extending this pattern to the read path (recall_many()) brings parity and symmetry to the memory architecture.
  • In multi-query agent workflows, single-call batch embedding cuts memory retrieval latency by 60% to 80%.
Willingness to Contribute

Yes, I'd be happy to submit a pull request

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.