GoogleCloudPlatform / GoogleCloudPlatform/firebase-extensions
firestore-vector-search: embedder model changes don't trigger a re-embed (backfill ignores model id)
- Dominant language
- TypeScript
- Stars
- 124
- Forks
- 67
- Avg merge
- 5d 5h
- Merged PRs (30d)
- 5
Description
### Problem
`shouldDoBackfill` (functions/src/index.ts:55) compares provider, dimension, inputField and outputField against the stored backfill metadata, but not the embedder model id. Both our embedder swaps so far (text-embedding-004 -> gemini-embedding-001 in #781, and the proposed gemini-embedding-001 -> gemini-embedding-2 in #1133) keep provider and dimension (768) identical, so upgrades never re-embed existing documents.
Result: stored vectors remain in the old model's embedding space while new writes and query-time embeddings use the new model. Nearest-neighbor search degrades silently, no errors. Installs that upgraded through 0.1.0 are already in this state.
### Proposed fix
1. Include the embedder model id in the stored backfill metadata and in the `shouldDoBackfill` comparison, so a model change triggers a full re-embed on upgrade (the comment above `shouldDoBackfill` already says "provider or model").
2. Then re-apply the gemini-embedding-2 swap (removed from #1133), which will also re-embed the mixed-space installs from #781. Before that lands, verify the `gemini-embedding-2` id resolves on the Vertex AI provider path, not just the Gemini API.
No urgency from deprecation: gemini-embedding-001 retirement is May 2028 (worth verifying).
### Refs
- #1133 (review discussion)
- #781 (previous swap with the same gap)
Contributor guide
Research direction
Start in functions/src/index.ts at shouldDoBackfill and trace where the stored backfill metadata is created and compared. Confirm that the embedder model id participates in backfill detection, then inspect the Vertex AI provider path to verify gemini-embedding-2 resolves there before reapplying the swap described in #1133.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, typescript
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 50/100