ADORSYS-GIS / ADORSYS-GIS/lightbridge-code-intelligence

[Ticket]: Batch upsert_code_chunks the same way UNWIND batches upsert_graph

Đang mở
#630 0 bình luận 0 reaction 1 người được giao Được @leghadjeu-christian nhận Xem trên GitHub
ticket
Ngôn ngữ chính
Rust
Star
0
Fork
0
Merge trung bình
14 giờ 13 phút
Pull request đã merge (30 ngày)
16

Mô tả

### Type
Performance

### Summary
We need to batch `upsert_code_chunks`'s writes into Postgres because it currently issues one
`INSERT ... ON CONFLICT DO UPDATE` per chunk, sequentially, inside a single transaction — the same
per-row-round-trip anti-pattern `upsert_graph` had against Neo4j before #626 batched it with
`UNWIND`.

Expected result: a repo's chunk batch is written to `code_chunks` in a small, bounded number of
round trips instead of one per chunk.

### Intent
`services/control-plane/src/db/code_chunks.rs::upsert_code_chunks` loops over every chunk and
executes its own `INSERT` inside one transaction. For a large repo (thousands of chunks — the same
class of repo that motivated #626's Neo4j fix), this is thousands of sequential round trips held
open in one transaction, inside one HTTP request the runner's `ControlPlaneClient` has no timeout
on. It's reachable from the exact same `index_checkout` flow as the already-fixed graph write.

### Source of truth (links)
Follow-up item from #626's review — see that PR's discussion and the implementation plan shared
alongside it (item 2, "Batch upsert_code_chunks the same way").

### Current Behavior
```rust
for chunk in chunks {
sqlx::query("INSERT INTO code_chunks (...) VALUES (...) ON CONFLICT (...) DO UPDATE SET ...")
.bind(...)
.execute(&mut *tx)
.await?;
}
```
One statement per chunk, sequentially, inside one transaction.

### Expected Behavior
A multi-row `INSERT ... VALUES (...), (...), ... ON CONFLICT (...) DO UPDATE SET ...` (built via
`sqlx::query_builder::QueryBuilder`, not currently used elsewhere in this codebase but already part
of the `sqlx` dependency already in use), issued in a small, bounded number of statements rather than
one per row. Same `ON CONFLICT` semantics, same public signature.

### Acceptance Criteria
- [ ] Given a batch of chunks, when `upsert_code_chunks` runs, then the same rows land in
`code_chunks` as today (insert-or-update semantics unchanged).
- [ ] Given a very large chunk count, when `upsert_code_chunks` runs, then rows are still batched in
bounded-size groups (not one unbounded statement) — mirrors why `upsert_graph`'s `UNWIND` payload
size is a known, accepted tradeoff only at today's observed graph sizes.
- [ ] Existing chunk-upsert tests (idempotency / `ON CONFLICT` behavior) pass unchanged.
- [ ] The `::vector` cast + `ON CONFLICT DO UPDATE` interaction is specifically covered by a test.

### Out of Scope
- Any change to `upsert_graph`/Neo4j (already done in #626).
- A request timeout on `ControlPlaneClient` — tracked separately.
- Running the live-Neo4j tests in CI — not part of this ticket.

### Technical Context
`services/control-plane/src/db/code_chunks.rs::upsert_code_chunks`, called from
`services/control-plane/src/http/internal.rs` (chunk-ingest endpoint), reachable from the same
per-task `index_checkout` flow as `upsert_graph`.

### Risks
Low-medium — the `ON CONFLICT` + `::vector` cast interaction is the one part worth testing
carefully; otherwise follows the same shape as #626's Neo4j batching.

### Test Plan
Existing `code_chunks` test suite, plus a new test with a chunk count large enough to exercise any
batch-size cap.

### Verification evidence
Test output: see the eventual PR.

### Human accountable owner
@leghadjeu-christian

### AI Usage Declaration
Drafting the ticket, Understanding code, Proposing implementation

### Human verification completed
- [x] I am the accountable owner and accept responsibility for this ticket.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.