Plan preview request routing scans all active pipeds
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 1.4k
- Forks
- 364
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 84
Description
What happened:
RequestPlanPreview currently lists all enabled pipeds in the project and scans every configured repository entry in memory to find pipeds that match the requested repository remote and base branch. This makes each interactive plan preview request scale with total active piped count and repository mappings instead of the number of matching pipeds.
Summary
Plan preview request routing performs an O(number of pipeds x repositories per piped) in-memory scan for every request.
Affected Area
pkg/app/server/grpcapi/api.go, specifically API.RequestPlanPreview.
Steps to Reproduce
- Create a test or benchmark environment with many active pipeds in one project and repository mappings on those pipeds.
- Call
API.RequestPlanPreviewrepeatedly for one repository remote and base branch. - Capture allocations and latency with
go test -benchor pprof. - Observe that request cost grows with total piped count rather than with the matching pipeds.
What you expected to happen:
Plan preview routing should avoid a full active-piped scan on every request. Repeated routing for a repository/base-branch pair should be close to O(matches), for example via an index, bounded cache, or targeted lookup.
Expected Behaviour
Routing a plan preview request should avoid repeated full scans and should keep behavior correct when multiple pipeds match the same repository and branch.
How to reproduce it:
- Seed or mock thousands of enabled pipeds for one project.
- Give only a subset a repository entry matching the plan preview request remote and base branch.
- Repeatedly call
RequestPlanPreviewfor that remote and branch. - Compare latency/allocations as piped count increases.
Actual Behaviour
Each request calls pipedStore.List with only ProjectId and Disabled=false filters, then iterates through every returned piped and every repository entry until it finds matching mappings.
Technical Evidence
In pkg/app/server/grpcapi/api.go, RequestPlanPreview lists enabled pipeds for the project and then builds a repositories map by checking each p.Repositories entry against req.RepoRemoteUrl and req.BaseBranch. The code also has a TODO noting that caching may be needed to reduce database load.
Suspected Root Cause
The API layer has no repository-to-piped routing index or cache, so it must perform repository matching after fetching all enabled pipeds.
Proposed Direction
Add a focused benchmark for plan preview routing at small and large piped cardinalities, then avoid repeated full scans by adding a bounded cache or another targeted lookup for repository-to-piped matches.
Acceptance Criteria
- A benchmark covering small and large piped counts is added.
- Repeated plan preview routing for the same project/repository/base-branch avoids repeated full active-piped scans.
- Correct behavior is preserved when multiple pipeds match the same repository and branch.
Environment:
pipedversion: Not applicablecontrol-planeversion: Currentmasterbranch- Others: Not applicable
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in pkg/app/server/grpcapi/api.go at API.RequestPlanPreview and inspect the pipedStore.List call plus repository-matching loop. Add a benchmark covering small and large piped counts, then evaluate a targeted lookup, index, or bounded cache against repeated requests. Done means repeated routing avoids full active-piped scans while preserving correct results when multiple pipeds match.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100