debezium / debezium/dbz

Standardize atomic upsert semantics and error handling across vector store adapters

Open
#2,618 0 comments 0 reactions 0 assignees View on GitHub
component/ai-python type/task
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

### Description
This issue improves vector store adapter reliability and safety in `pydebeziumai`. It standardizes atomic upsert semantics across all adapters to prevent data loss during transient embedding failures, improves `MilvusAdapter` error handling and connection management, and clarifies adapter interface contracts.

1. **Atomic Vector Upserts & Data Loss Prevention**:
- Update `ChromaAdapter.upsert` and `MilvusAdapter.upsert` to eliminate non-atomic `delete(id)` + `add_documents(...)` execution steps.
- Use native atomic upserts or direct `add_documents` calls to prevent documents from being deleted if subsequent embedding calls fail.
2. **Milvus Adapter Resiliency & Connection Safety**:
- Catch missing collection `MilvusException` (`code == 100`) inside `MilvusAdapter.delete()` and handle it gracefully as a no-op.
- Hash `connection_uri` into Milvus connection aliases (`f"pydebeziumai_{hash(uri)}"`) to prevent multi-instance alias collisions.
- Add strict type assertions in `_dict_to_milvus_expr` to prevent string format expression fallthroughs.
3. **Interface Contracts & Docstring Accuracy**:
- Update `VectorStoreAdapter` base class docstring requiring single atomic upserts.
- Update `PGVectorAdapter` docstring clarifying sync and async `postgresql+psycopg://` URL compatibility.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.