AB-Law / AB-Law/Pluck-It

`scraped_items.py`: In-memory vector similarity instead of Azure AI Search

Open
#73 1 comment 0 reactions 1 assignee Claimed by @AB-Law View on GitHub
Dominant language
TypeScript
Stars
0
Forks
0
Avg merge
16m
Merged PRs (30d)
1

Description

**File:** `PluckIt.Processor/agents/tools/scraped_items.py:87`

### Problem

`search_scraped_items` loads 200 candidate documents from Cosmos — including their full embedding vectors — and computes cosine similarity in Python:

```python
_CANDIDATE_LIMIT = 200 # how many docs to load for in-memory ranking

async for doc in container.query_items(
query="... ORDER BY c.scoreSignal DESC OFFSET 0 LIMIT 200"
):
candidates.append(doc)

for doc in candidates:
emb = doc.get("embedding") or []
sim = _cosine_similarity(query_embedding, emb)
```

This approach has two compounding problems:
1. **200 Cosmos reads per search** — each document includes its full embedding vector (~6KB for `text-embedding-3-small`). That's ~1.2 MB of data transferred and ~2,000 RU per tool call.
2. **Candidate selection is by `scoreSignal`, not semantic relevance** — the 200 items sent for ranking are chosen by a scraper quality score, not by closeness to the query. A semantically perfect match ranked 201st is invisible.

### Impact

| Users | Stylist sessions/month | Scraped-item searches | Monthly Cosmos RU cost |
|-------|----------------------|----------------------|----------------------|
| 10 | 300 | ~300 | ~600K RU (negligible) |
| 100 | 3,000 | ~3,000 | ~6M RU — requires autoscale |
| 1,000 | 30,000 | ~30,000 | ~60M RU — ~$90/month in Cosmos alone |

At 100+ users this becomes the single largest Cosmos cost driver.

### Proposed Fix

Move similarity search to **Azure AI Search** with a vector index on the `embedding` field. The query becomes a single ANN (approximate nearest-neighbour) search call returning the top-K results directly, with no client-side scoring loop and no full-vector document reads from Cosmos.

This is already the infrastructure direction implied by the Azure stack — Azure AI Search is already provisioned in the Terraform config.

### Functionality Impact

Results improve: ANN search considers the full corpus rather than the top-200 by score signal, so relevant items that scored low on the scraper heuristic are no longer invisible. Latency also drops — one search API call vs 200 Cosmos reads.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.