ADORSYS-GIS / ADORSYS-GIS/lightbridge-code-intelligence
[Ticket]: Batch upsert_code_chunks the same way UNWIND batches upsert_graph
- 主要语言
- Rust
- 星标
- 0
- 派生
- 0
- 平均合并
- 14 小时 13 分钟
- 30 天内合并 PR
- 16
描述
### Type
Performance
### Summary
We need to batch `upsert_code_chunks`'s writes into Postgres because it currently issues one
`INSERT ... ON CONFLICT DO UPDATE` per chunk, sequentially, inside a single transaction — the same
per-row-round-trip anti-pattern `upsert_graph` had against Neo4j before #626 batched it with
`UNWIND`.
Expected result: a repo's chunk batch is written to `code_chunks` in a small, bounded number of
round trips instead of one per chunk.
### Intent
`services/control-plane/src/db/code_chunks.rs::upsert_code_chunks` loops over every chunk and
executes its own `INSERT` inside one transaction. For a large repo (thousands of chunks — the same
class of repo that motivated #626's Neo4j fix), this is thousands of sequential round trips held
open in one transaction, inside one HTTP request the runner's `ControlPlaneClient` has no timeout
on. It's reachable from the exact same `index_checkout` flow as the already-fixed graph write.
### Source of truth (links)
Follow-up item from #626's review — see that PR's discussion and the implementation plan shared
alongside it (item 2, "Batch upsert_code_chunks the same way").
### Current Behavior
```rust
for chunk in chunks {
sqlx::query("INSERT INTO code_chunks (...) VALUES (...) ON CONFLICT (...) DO UPDATE SET ...")
.bind(...)
.execute(&mut *tx)
.await?;
}
```
One statement per chunk, sequentially, inside one transaction.
### Expected Behavior
A multi-row `INSERT ... VALUES (...), (...), ... ON CONFLICT (...) DO UPDATE SET ...` (built via
`sqlx::query_builder::QueryBuilder`, not currently used elsewhere in this codebase but already part
of the `sqlx` dependency already in use), issued in a small, bounded number of statements rather than
one per row. Same `ON CONFLICT` semantics, same public signature.
### Acceptance Criteria
- [ ] Given a batch of chunks, when `upsert_code_chunks` runs, then the same rows land in
`code_chunks` as today (insert-or-update semantics unchanged).
- [ ] Given a very large chunk count, when `upsert_code_chunks` runs, then rows are still batched in
bounded-size groups (not one unbounded statement) — mirrors why `upsert_graph`'s `UNWIND` payload
size is a known, accepted tradeoff only at today's observed graph sizes.
- [ ] Existing chunk-upsert tests (idempotency / `ON CONFLICT` behavior) pass unchanged.
- [ ] The `::vector` cast + `ON CONFLICT DO UPDATE` interaction is specifically covered by a test.
### Out of Scope
- Any change to `upsert_graph`/Neo4j (already done in #626).
- A request timeout on `ControlPlaneClient` — tracked separately.
- Running the live-Neo4j tests in CI — not part of this ticket.
### Technical Context
`services/control-plane/src/db/code_chunks.rs::upsert_code_chunks`, called from
`services/control-plane/src/http/internal.rs` (chunk-ingest endpoint), reachable from the same
per-task `index_checkout` flow as `upsert_graph`.
### Risks
Low-medium — the `ON CONFLICT` + `::vector` cast interaction is the one part worth testing
carefully; otherwise follows the same shape as #626's Neo4j batching.
### Test Plan
Existing `code_chunks` test suite, plus a new test with a chunk count large enough to exercise any
batch-size cap.
### Verification evidence
Test output: see the eventual PR.
### Human accountable owner
@leghadjeu-christian
### AI Usage Declaration
Drafting the ticket, Understanding code, Proposing implementation
### Human verification completed
- [x] I am the accountable owner and accept responsibility for this ticket.
贡献指南
评估
这个 Issue 还没有评估数据。