cloudflare / cloudflare/serverless-registry
api route /v2/_catalog slow
- Dominant language
- TypeScript
- Stars
- 1.5k
- Forks
- 122
- PR merge metrics
- No merged PRs in 30d
Description
Hi, it seems that for listing all repositories the whole r2 bucket is scanned:
https://github.com/cloudflare/serverless-registry/blob/main/src/registry/r2.ts#L163
This doesn't scale well and in our case (5 repositories with +-100 tags each), the listing already takes +-10 seconds. Looking into the code and r2 api, i can't actually think of some optimization how to simply do that with current approach. Maybe creating extra placeholder objcts in r2 like: `repositories/repo1` `repositories/repo2` will help scanning just this `repositories/*` prefix.
Contributor guide
Research direction
Start at src/registry/r2.ts#L163, which handles listing for the /v2/_catalog route, and read the relevant R2 API behavior. Reproduce the delay with several repositories and roughly 100 tags per repository, then measure whether the listing avoids scanning the whole bucket. Done means the catalog response is returned substantially faster at that scale without changing its results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, backend, cloud, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100