Basekick-Labs / Basekick-Labs/arc

arcx Arrow IPC serve path applies no governance row cap and registers no trailers

Open
#727 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
677
Forks
53
Avg merge
9h 14m
Merged PRs (30d)
164

Description

Follow-up from #724.

`tryArcxRouterArrow` (`internal/api/arcx_hook.go`) takes no `governanceMaxRows` parameter and its IPC loop applies no cap, which `internal/api/query_arrow.go:541-543` already documents:

> the MaxRows cap is enforced only by the DuckDB IPC loop below, the experimental arcx serve path streams uncapped until it learns to take a row cap.

#724 adds an `Arc-Rows-Capped` trailer to the DuckDB Arrow IPC path. That does not reach this path either: `tryArcxRouterArrow` returns at `query_arrow.go:544`, well before the `AddTrailer` block at `query_arrow.go:605-624`, so an arcx-served Arrow IPC response registers none of the three trailers (`Arc-Execution-Time-Ms` and `Arc-Stream-Truncated` from #722 are equally absent).

Net effect for a client in `arcx_engine` builds: no cap is applied, and no trailer says so, so "cap not reached" and "cap not applied" are indistinguishable. #724's release note tells clients to read a missing trailer as "unknown" rather than "not capped", which is a workaround, not a fix.

Both halves want fixing together:

1. Thread the governance cap into `tryArcxRouterArrow` and enforce it in its IPC loop, matching `streamArrowIPC`.
2. Register the three trailers before the arcx hook runs, or register them inside the arcx serve path, so an arcx-served response carries the same signals.

Only affects builds with the `arcx_engine` tag, which no CI or release build currently uses. Note that `internal/api/arcx_hook.go` is not compile-checked by CI for the same reason.

Contributor guide

Open the contributing guide

Research direction

Start with tryArcxRouterArrow in internal/api/arcx_hook.go, then compare its IPC loop with streamArrowIPC and the trailer registration block in internal/api/query_arrow.go:541-543,605-624. Trace the arcx_engine-tagged serve path first. Done means the path enforces governanceMaxRows and its response registers Arc-Execution-Time-Ms, Arc-Stream-Truncated, and Arc-Rows-Capped.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.