cloudflare / cloudflare/serverless-registry

api route /v2/_catalog slow

Open
#104 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1.5k
Forks
122
PR merge metrics
No merged PRs in 30d

Description

Hi, it seems that for listing all repositories the whole r2 bucket is scanned:

https://github.com/cloudflare/serverless-registry/blob/main/src/registry/r2.ts#L163

This doesn't scale well and in our case (5 repositories with +-100 tags each), the listing already takes +-10 seconds. Looking into the code and r2 api, i can't actually think of some optimization how to simply do that with current approach. Maybe creating extra placeholder objcts in r2 like: `repositories/repo1` `repositories/repo2` will help scanning just this `repositories/*` prefix.

Contributor guide

Open the contributing guide

Research direction

Start at src/registry/r2.ts#L163, which handles listing for the /v2/_catalog route, and read the relevant R2 API behavior. Reproduce the delay with several repositories and roughly 100 tags per repository, then measure whether the listing avoids scanning the whole bucket. Done means the catalog response is returned substantially faster at that scale without changing its results.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api, backend, cloud, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.