GoogleCloudPlatform / GoogleCloudPlatform/firebase-extensions

fix(firestore-incremental-capture): vector fields restore as plain maps, silently breaking vector search

Open
#1,151 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
124
Forks
67
Avg merge
5d 5h
Merged PRs (30d)
5

Description

### [REQUIRED] Step 2: Describe your configuration

- Extension name: `firestore-incremental-capture` (and the kit port on firebase/extensions `kits` branch)
- Extension version: any; affects the serializer on the capture side and the `firestore-incremental-capture-pipeline` reconstructor
- Configuration values: not configuration dependent - affects every installation whose documents contain Firestore vector fields (`FieldValue.vector(...)`)

### [REQUIRED] Step 3: Describe the problem

The serializer has no case for `VectorValue`, so a vector field falls into the generic object branch and is captured as a plain map that leaks the class's private `_values` property:

```json
{"type": "map", "value": {"_values": {"type": "array", "value": [...]}}}
```

The pipeline faithfully restores that as a literal map `{_values: [1.5, 2.5]}`. The restored document has a map where the vector was, so vector search over restored data is silently broken. The job reports success and logs nothing.

Verified against the Firestore emulator (write `FieldValue.vector([1.5, 2.5])`, capture with the kit serializer, restore with the pipeline at `fix/fic-pipeline-array-elements`). Present in the kit serializer; the legacy extension serializer has no `VectorValue` case either.

#### Steps to reproduce:

1. Install the extension or kit and let the changelog accumulate.
2. Write a document with a vector field after the backup baseline.
3. Run a restoration for a timestamp after the write.

##### Expected result

The field restores as a Firestore vector.

##### Actual result

The field restores as a plain map `{_values: [...]}`.

#### Impact

Silent corruption of the disaster-recovery path for any collection using vector search. Not listed in the kit README's known limitations.

#### Suggested fix

Two-sided, same family as #1147:

- Serializer: detect `VectorValue` and emit a dedicated tag, e.g. `{type: "vector", value: [numbers]}`, instead of recursing into the object.
- Reconstructor: add a `VECTOR` case building Firestore's wire representation (a `mapValue` with `__type__: "__vector__"` and a `value` array), and keep skipping the old leaked shape gracefully for changelogs already in place (those rows captured only `_values`, so the numbers are recoverable if worth the compatibility code).

#### Related

Found while verifying #1150. Same silent-loss family as #1147; like it, pre-existing in both the extension and the kit port.

Contributor guide

Open the contributing guide

Research direction

Start by locating the firestore-incremental-capture serializer and the firestore-incremental-capture-pipeline reconstructor, then reproduce the issue with the Firestore emulator using FieldValue.vector([1.5, 2.5]). Trace the generic object serialization and restoration paths. Done means vector fields restore as Firestore vectors, both the extension and kit serializer are covered, and legacy _values changelog entries are handled or explicitly tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.