Basekick-Labs / Basekick-Labs/arc
arcx Arrow IPC serve path applies no governance row cap and registers no trailers
- Dominant language
- Go
- Stars
- 677
- Forks
- 53
- Avg merge
- 9h 14m
- Merged PRs (30d)
- 164
Description
Follow-up from #724.
`tryArcxRouterArrow` (`internal/api/arcx_hook.go`) takes no `governanceMaxRows` parameter and its IPC loop applies no cap, which `internal/api/query_arrow.go:541-543` already documents:
> the MaxRows cap is enforced only by the DuckDB IPC loop below, the experimental arcx serve path streams uncapped until it learns to take a row cap.
#724 adds an `Arc-Rows-Capped` trailer to the DuckDB Arrow IPC path. That does not reach this path either: `tryArcxRouterArrow` returns at `query_arrow.go:544`, well before the `AddTrailer` block at `query_arrow.go:605-624`, so an arcx-served Arrow IPC response registers none of the three trailers (`Arc-Execution-Time-Ms` and `Arc-Stream-Truncated` from #722 are equally absent).
Net effect for a client in `arcx_engine` builds: no cap is applied, and no trailer says so, so "cap not reached" and "cap not applied" are indistinguishable. #724's release note tells clients to read a missing trailer as "unknown" rather than "not capped", which is a workaround, not a fix.
Both halves want fixing together:
1. Thread the governance cap into `tryArcxRouterArrow` and enforce it in its IPC loop, matching `streamArrowIPC`.
2. Register the three trailers before the arcx hook runs, or register them inside the arcx serve path, so an arcx-served response carries the same signals.
Only affects builds with the `arcx_engine` tag, which no CI or release build currently uses. Note that `internal/api/arcx_hook.go` is not compile-checked by CI for the same reason.
Contributor guide
Research direction
Start with tryArcxRouterArrow in internal/api/arcx_hook.go, then compare its IPC loop with streamArrowIPC and the trailer registration block in internal/api/query_arrow.go:541-543,605-624. Trace the arcx_engine-tagged serve path first. Done means the path enforces governanceMaxRows and its response registers Arc-Execution-Time-Ms, Arc-Stream-Truncated, and Arc-Rows-Capped.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100