firebase / firebase/extensions

firestore-vector-search: multimodal embedding provider is non-functional in both extension and kit

Open
#3,135 1 comment 0 reactions 0 assignees View on GitHub
type: bug
Dominant language
TypeScript
Stars
979
Forks
433
Avg merge
1d 22h
Merged PRs (30d)
127

Description

`EMBEDDING_PROVIDER=multimodal` is offered by both the extension and the kit, but neither produces working embeddings:

- **Extension** ([firestore-vector-search](https://github.com/GoogleCloudPlatform/firebase-extensions/tree/main/firestore-vector-search)): [`MultimodalEmbeddingClient`](https://github.com/GoogleCloudPlatform/firebase-extensions/blob/main/firestore-vector-search/functions/src/embeddings/client/multimodal/index.ts) targets `locations/us-central-1` (invalid region), hardcodes `image/jpeg`, and has no response error handling.
- **Kit** (`kits/firestore-vector-search`): `MultimodalEmbedClient` throws `Multimodal embeddings are not implemented in this package`.

Selecting it deploys an instance that fails on every embedding.

**Options:** implement it properly in both, or remove the value from both. Removing is a breaking config change, so it is deferred until the parity work is done (parity ledger #2974, firestore-vector-search §3).

Related: #3014, #3074 (closed, not pursued for now).

Contributor guide

Open the contributing guide

Research direction

Start by reading the extension's functions/src/embeddings/client/multimodal/index.ts and the kit's MultimodalEmbedClient, then consult parity ledger #2974 §3 and related issues. Trace the failure in both packages and determine the agreed parity path. Done means multimodal selection no longer deploys an instance that fails every embedding, consistently across the extension and kit.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, databases, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.