vimcolorschemes / vimcolorschemes/worker
parallelize update github reads
Open
Nobody has claimed this yet.
enhancement
- Dominant language
- Go
- Stars
- 5
- Forks
- 0
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 4
Description
the update job still fetches repositories from github one at a time. after DB writes are batched, github/network latency will likely become the next major bottleneck.
- add a small worker pool for github repository reads, likely 5-10 workers
- preserve existing 404 delete handling and error counts
- handle primary and secondary github rate limits with backoff
- keep database writes batched after results are collected
- verify update duration and github error behavior in ecs logs
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the update job and the existing GitHub repository reads, 404 deletion handling, error counting, and batched database writes. Review current ECS logs for update duration and GitHub errors before changing concurrency; done means reads use a small worker pool, rate limits back off, writes remain batched, and logs show preserved error behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, github, go
- Domain
- backend, cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100