GoogleCloudPlatform / GoogleCloudPlatform/firebase-extensions
fix(firestore-incremental-capture): vector fields restore as plain maps, silently breaking vector search
- Dominant language
- TypeScript
- Stars
- 124
- Forks
- 67
- Avg merge
- 5d 5h
- Merged PRs (30d)
- 5
Description
### [REQUIRED] Step 2: Describe your configuration
- Extension name: `firestore-incremental-capture` (and the kit port on firebase/extensions `kits` branch)
- Extension version: any; affects the serializer on the capture side and the `firestore-incremental-capture-pipeline` reconstructor
- Configuration values: not configuration dependent - affects every installation whose documents contain Firestore vector fields (`FieldValue.vector(...)`)
### [REQUIRED] Step 3: Describe the problem
The serializer has no case for `VectorValue`, so a vector field falls into the generic object branch and is captured as a plain map that leaks the class's private `_values` property:
```json
{"type": "map", "value": {"_values": {"type": "array", "value": [...]}}}
```
The pipeline faithfully restores that as a literal map `{_values: [1.5, 2.5]}`. The restored document has a map where the vector was, so vector search over restored data is silently broken. The job reports success and logs nothing.
Verified against the Firestore emulator (write `FieldValue.vector([1.5, 2.5])`, capture with the kit serializer, restore with the pipeline at `fix/fic-pipeline-array-elements`). Present in the kit serializer; the legacy extension serializer has no `VectorValue` case either.
#### Steps to reproduce:
1. Install the extension or kit and let the changelog accumulate.
2. Write a document with a vector field after the backup baseline.
3. Run a restoration for a timestamp after the write.
##### Expected result
The field restores as a Firestore vector.
##### Actual result
The field restores as a plain map `{_values: [...]}`.
#### Impact
Silent corruption of the disaster-recovery path for any collection using vector search. Not listed in the kit README's known limitations.
#### Suggested fix
Two-sided, same family as #1147:
- Serializer: detect `VectorValue` and emit a dedicated tag, e.g. `{type: "vector", value: [numbers]}`, instead of recursing into the object.
- Reconstructor: add a `VECTOR` case building Firestore's wire representation (a `mapValue` with `__type__: "__vector__"` and a `value` array), and keep skipping the old leaked shape gracefully for changelogs already in place (those rows captured only `_values`, so the numbers are recoverable if worth the compatibility code).
#### Related
Found while verifying #1150. Same silent-loss family as #1147; like it, pre-existing in both the extension and the kit port.
Contributor guide
Research direction
Start by locating the firestore-incremental-capture serializer and the firestore-incremental-capture-pipeline reconstructor, then reproduce the issue with the Firestore emulator using FieldValue.vector([1.5, 2.5]). Trace the generic object serialization and restoration paths. Done means vector fields restore as Firestore vectors, both the extension and kit serializer are covered, and legacy _values changelog entries are handled or explicitly tested.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100