Memory recall dedup keeps whichever duplicate a thread finished first, not the best score

Description

RecallFlow.synthesize_results deduplicates by keeping the first occurrence of each record.id, but the findings it iterates arrive in thread-completion order. When the same record is reachable from more than one (sub-query, scope) search task, which of its scores survives is decided by thread timing, so the same query can return the same record at a different rank on consecutive runs.

lib/crewai/src/crewai/memory/recall_flow.py, synthesize_results:

seen_ids: set[str] = set()
matches: list[MemoryMatch] = []
for finding in self.state.chunk_findings:
    ...
    for item in results:
        ...
        if isinstance(record, MemoryRecord) and record.id not in seen_ids:
            seen_ids.add(record.id)          # <-- first wins
            composite, reasons = compute_composite_score(record, float(score), self._config)
            matches.append(MemoryMatch(record=record, score=composite, match_reasons=reasons))
matches.sort(key=lambda m: m.score, reverse=True)
final_results = matches[: self.state.limit]

And _do_search, which produces chunk_findings:

with ThreadPoolExecutor(max_workers=min(len(tasks), 4)) as pool:
    futures = {pool.submit(...): (emb, sc) for emb, sc in tasks}
    for future in as_completed(futures):     # <-- completion order, not task order
        ...
        findings.append({"scope": scope, "results": results, "top_score": top_composite})

Why duplicates are guaranteed, not hypothetical

Two independent reasons, both structural:

1. Scope prefix matching overlaps by design. lib/crewai/src/crewai/memory/storage/lancedb_storage.py:

   if scope_prefix is not None and scope_prefix.strip("/"):
       prefix = scope_prefix.rstrip("/")
       like_val = prefix + "%"
       query = query.where(f"scope LIKE '{like_val}'")
   

   A record stored at /a/b is returned for scope /a and for scope /a/b. filter_and_chunk routinely selects both (selected_scopes = candidates[:20]).

2. Each sub-query embedding scores the same record differently. analyze_query_step produces up to 3 sub-query embeddings, and _do_search runs the full embeddings × scopes cross product — up to 60 tasks against overlapping scopes.

So the same record.id legitimately arrives from several tasks carrying different semantic scores, and compute_composite_score is monotonic in semantic_score:

composite = (
    config.semantic_weight * semantic_score
    + config.recency_weight * decay
    + config.importance_weight * record.importance
)

Keeping the first-arriving one therefore does not merely reorder — it can keep a strictly worse score and under-rank the record, or push it out of matches[: self.state.limit] entirely. The duplicate also consumed nothing visible, but the lower score it left behind is what the caller ranks on.

Reproduction

Standalone, no storage backend needed — it models _do_search's dispatch plus the first-wins dedup, and only the sleep order differs between the two runs:

"""Models _do_search's dispatch plus synthesize_results' first-wins dedup.

One record 'R' lives at /a/b, so lancedb's `scope LIKE 'prefix%'` returns it for
scope /a AND for scope /a/b, and each sub-query embedding scores it differently.
Only the per-task delays differ between the two runs below.
"""

from concurrent.futures import ThreadPoolExecutor, as_completed
import time


def run(tasks):
    """tasks: list of (scope, semantic_score, delay_seconds)."""

    def one(scope, score, delay):
        time.sleep(delay)
        return {"scope": scope, "results": [("R", score)]}

    findings = []
    with ThreadPoolExecutor(max_workers=2) as pool:
        futures = [pool.submit(one, *t) for t in tasks]
        for future in as_completed(futures):
            findings.append(future.result())

    seen, out = set(), []
    for finding in findings:
        for rid, score in finding["results"]:
            if rid not in seen:          # first wins -- the bug
                seen.add(rid)
                out.append((rid, score, finding["scope"]))
    return out


print("record 'R': scope /a scores 0.9, scope /a/b scores 0.4")
print("  /a finishes first: ", run([("/a", 0.9, 0.00), ("/a/b", 0.4, 0.05)]))
print("  /a/b finishes first:", run([("/a", 0.9, 0.05), ("/a/b", 0.4, 0.00)]))

Measured output:

record 'R': scope /a scores 0.9, scope /a/b scores 0.4
  /a finishes first:  [('R', 0.9, '/a')]
  /a/b finishes first: [('R', 0.4, '/a/b')]

Same inputs, different surviving score, purely from which thread finished first.

Impact on callers

UnifiedMemory consumes flow.state.final_results directly (lib/crewai/src/crewai/memory/unified_memory.py), then calls touch_records on it and emits MemoryQueryCompletedEvent, so the nondeterminism reaches user-visible recall results, last_accessed bookkeeping, and telemetry.

There is a second, smaller ordering dependence: matches.sort(key=lambda m: m.score, reverse=True) is stable, so records with equal composite scores keep their arrival order — also as_completed order.

Suggested fix

Keep the best score per record.id rather than the first, and give the sort a deterministic tiebreak on record.id. Best-wins is both order-independent and the score ranking should be using anyway.

Out of scope

self.state.confidence = max((f["top_score"] for f in findings), default=0.0) in _do_search is already order-independent (max over the whole list) and needs no change.

Environment

• crewAI main
• Python 3.12, Windows 11

I have a fix plus regression tests ready and will open a PR referencing this issue.

Start in lib/crewai/src/crewai/memory/recall_flow.py, reading synthesize_results and _do_search, then inspect the overlapping scope behavior in lib/crewai/src/crewai/memory/storage/lancedb_storage.py. Review the regression tests mentioned in the issue and verify that duplicate records retain the best score and that equal-score results have deterministic ordering. Check lib/crewai/src/crewai/memory/unified_memory.py to understand the caller-visible result flow.