chainguard-dev / chainguard-dev/apko
resolver cache retains a full package map per index combination; bound it by size or share the entries
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 1.7k
- Forks
- 228
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 62
Description
globalResolverCache (pkg/apk/apk/shameful_global_caches.go:94) holds up to 64 PkgResolvers, one per distinct index combination. Each newPkgResolver (pkg/apk/apk/resolver.go) builds its own nameMap and installIfMap over every package in its index set, allocating a fresh repositoryPackage wrapper per package and again per provides entry. Nothing is shared between resolvers built over overlapping indexes.
Before #2462 the currency gate discarded most resolvers, so this was cheap by accident. #2462 correctly made them cacheable, and the memory followed. In a long-running process that resolves against many distinct repository sets, a 24-hour averaged heap profile shows newPkgResolver at 1.8 GB in-use, 44% of the heap, up from 0.6 GB in the week before #2462 shipped. The 64-entry cap is by count, so a full cache over a large package universe costs whatever 64 resolvers happen to weigh.
Two directions, not exclusive:
- Bound by bytes, or lower the entry cap. Straightforward, but trades resolver rebuilds back in.
- Share the per-package wrappers.
repositoryPackageis{*RepositoryPackage, pinnedName}; wrappers for the same package and pin are identical across resolvers and could come from a per-index interned set, so each resolver only owns its maps. That keeps the CPU win from #2462 and shrinks each entry rather than evicting it.
Measured, not derived: numbers above are from Cloud Profiler heap profiles of one deployment, averaged across instances. The process-level number will vary with the size of the package universe and the number of distinct index sets.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with globalResolverCache in pkg/apk/apk/shameful_global_caches.go:94 and newPkgResolver in pkg/apk/apk/resolver.go. Compare a byte or entry bound with sharing repositoryPackage wrappers for overlapping indexes, using the reported heap profile as the baseline. Done means resolver memory is reduced while retaining the CPU benefit of the cache.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100