[Feature] Bound template mmap cache size and add memory-pressure eviction to prevent unbounded RSS growth
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 1.6k
- Forks
- 438
- PR merge metrics
- No merged PRs in 30d
Description
Problem
In self-hosted deployments with sustained sandbox workloads, the Orchestrator's RSS grows monotonically and is never reclaimed automatically.
Observed metrics: RSS increased from ~200 MB to ~37 GB within 48 hours.
Most memory consumption comes from Pss_File, which is occupied by mmap-backed template caches.
Root Cause Analysis
- Template cache has no capacity limit
The template cache in pkg/sandbox/template/cache.go is initialized only with a fixed TTL, without maximum capacity restriction:
cache := ttlcache.New(
ttlcache.WithTTLstring, Template, // hardcoded 25 h
// no WithCapacity — cache grows without bound
)
getTemplateWithFetch calls cache.GetOrSet, which resets TTL on every access.
Templates used by long-running sandboxes will never be evicted.
Each cache entry holds an mmap.MMap handle, locking file pages in RAM until eviction. - Build cache eviction only checks disk usage (no memory pressure handling)
The eviction loop startDiskSpaceEviction in pkg/sandbox/build/cache.go only judges by disk usage (BuildCacheMaxUsagePercentage, default 85%).
If disk usage is under threshold but host memory is exhausted, no cache eviction will be triggered.
The build diff cache (DiffStore) and template mmap cache are two independent subsystems; neither supports memory-based eviction.
Proposed Changes
Add capacity limit for template cache
Add ttlcache.WithCapacity to limit template cache size. Use an environment variable (e.g. TEMPLATE_CACHE_MAX_ENTRIES) for configuration, following existing BuilderConfig style. Evict entries by LRU policy when reaching the cap.
Add memory pressure detection for build cache eviction
Extend startDiskSpaceEviction:
Read system memory status via unix.Sysinfo
Trigger deleteOldestFromCache when free memory is lower than a configurable threshold (e.g. CACHE_MIN_FREE_MEMORY_MB)
Make template TTL configurable
Replace the hardcoded 25h templateExpiration with an environment variable. The 25h default works for cloud environments, but is too long for memory-limited self-hosted nodes.
Notes
All above changes work as safety valves only. They will not affect normal running behavior on nodes with sufficient resources.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with pkg/sandbox/template/cache.go and the getTemplateWithFetch path, then inspect pkg/sandbox/build/cache.go and startDiskSpaceEviction. Trace the existing BuilderConfig-style environment configuration and cache eviction behavior. Done means template capacity and TTL are configurable, and build-cache eviction can respond to low free memory as well as disk usage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, infrastructure, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100