[FEATURE] Add batch query recall (recall_many) to UnifiedMemory with single-call embedding
Open
@Rohitkanithi is already working on this.
Since Sep 17, 2026.
feature-request
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
Feature Area
Performance optimization
Is your feature request related to a an existing bug? Please link it here.
NA
Describe the solution you'd like
Add a batch recall method Memory.recall_many(queries: list[str], ...) to crewai.memory.UnifiedMemory:
- Single-Call Batch Embedding: Instead of sequential HTTP requests for each query, embed all query strings simultaneously in a single API call using the embedder's batch API (e.g. _encode_batch(queries)). Every major provider (OpenAI, Gemini, Cohere, Voyage AI, Ollama) natively accepts a list of strings in one request.
- Concurrent Vector Search: Query the vector storage backend for each embedding in parallel.
- Unified Cross-Query Deduplication & Ranking: Combine candidate matches across queries, deduplicate by record.id, compute composite scores (semantic, recency, importance), and return the top-ranked list[MemoryMatch].
- Update RecallMemoryTool: Update RecallMemoryTool._run in crewai/tools/memory_tools.py to use self.memory.recall_many(queries) instead of looping sequentially.
Describe alternatives you've considered
Currently, RecallMemoryTool accepts a list of queries (queries: list[str]), but executes them in a sequential loop:
for query in queries:
matches = self.memory.recall(query, limit=20) # Sequential network round-trip each time
For N queries, this incurs N consecutive round-trips to the embedding API. With typical API latency of 250-400ms per request, searching 4 queries takes 1.2-1.6 seconds, during which the agent is blocked waiting for I/O.
Additional context
- UnifiedMemory already has batch embedding on the write path (_encode_batch() used by remember_many()). Extending this pattern to the read path (recall_many()) brings parity and symmetry to the memory architecture.
- In multi-query agent workflows, single-call batch embedding cuts memory retrieval latency by 60% to 80%.
Willingness to Contribute
Yes, I'd be happy to submit a pull request
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.