container-cache: emptyDir-backed cache never evicts because min_free reads the node filesystem

Open
#1,576 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
68/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
kubernetes, nginx

Research direction

Start in the chart templates that render proxy_cache_path and the cache volume mounts, tracing the storageClassName: emptydir branch and its sizeProxyGB, sizeGB, and freeProxyPct values. Verify the rendered nginx configuration for emptyDir and persistentVolumeClaim modes; done means emptyDir zones include the proposed max_size accounting while persistent-volume deployments remain unchanged.

Written by the indexing model from the issue text.

Description

Problem

With persistentVolumeClaim.storageClassName: emptydir the chart mounts the cache directories as emptyDir volumes with sizeLimit set from sizeProxyGB / sizeGB, and drives nginx eviction with min_free only (no max_size on any proxy_cache_path).

An emptyDir has no filesystem of its own. nginx's min_free check runs statfs on the cache directory, which returns the free space of the node's ephemeral-storage volume (tens of TiB on a GPU node). A watermark of freeProxyPct percent of sizeProxyGB is therefore never reached, nginx never evicts, and the directory grows until kubelet enforces the emptyDir sizeLimit and evicts the whole pod. The cache restarts cold.

Observed while evaluating node-local NVMe as the cache backend: the node volume was 23.95 TiB, sizeProxyGB 1500, freeProxyPct 15, giving min_free=225g, about 1 percent of the volume.

Expected

In emptyDir mode nginx should evict by its own accounting before the emptyDir limit is hit, keeping freeProxyPct as the single knob meaning "start evicting at (100 - pct) percent usage".

Proposed fix

When storageClassName is emptydir, add max_size=(100 - freeProxyPct)% of the zone's configured size to each proxy_cache_path, keeping min_free as is. PersistentVolume deployments are unchanged.

Dominant language
Go
Stars
218
Forks
72
Avg merge
1d 9h
Merged PRs (30d)
417

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/nvcf

All issues in NVIDIA/nvcf

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.