AB-Law / AB-Law/Pluck-It

`scraper_runner.py`: Failed embedding batches silently upsert zero-vector items

未关闭
#56 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
bug help wanted
主要语言
TypeScript
星标
0
派生
0
平均合并
16 分钟
30 天内合并 PR
1

描述

### Problem

When an embedding batch fails (Azure OpenAI timeout, quota exhaustion, etc.), `scraper_runner.py` fills the embedding field with an empty list `[]` and continues to upsert those items:

**File:** `PluckIt.Processor/agents/scraper_runner.py:219`

```python
embeddings = [] # fallback on failure
# items are still upserted below with empty embeddings
```

Items upserted with a zero/empty vector become silently unfindable via semantic search. There is no log of which batches failed or which item IDs are affected.

### Impact

- Scraped items that hit a transient embedding failure are permanently broken for vector search
- No observability — operators have no way to identify or reprocess affected items
- Silent data corruption: Cosmos has the item, but search never surfaces it

### Proposed Fix

1. Log failed item IDs with `logger.warning("Embedding failed for items: %s", item_ids)`
2. Skip the upsert for items with failed embeddings (or mark them with `embeddingStatus: "failed"`)
3. Optionally retry the batch with exponential backoff before giving up

### Functionality Impact

Items with failed embeddings are already broken for search — skipping their upsert prevents the silent corruption. A retry makes the scraper more resilient to transient quota limits.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。