nvsnap: replace environment-variable configuration with typed flags and chart values
@balajinvda is already working on this.
Since Aug 19, 2026.
- Dominant language
- Go
- Stars
- 218
- Forks
- 72
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 427
Description
Problem
nvsnap behaviour is configured through roughly 40 distinct NVSNAP_* environment
variables read across the agent, webhook, restore entrypoint and helper binaries.
Thirty of them appear nowhere in the Helm chart, so the only way to learn what a
running cluster is set to is to exec into a DaemonSet and read the pod spec.
This is not theoretical. During the recent restore-reliability investigation
(#925) time was lost suspecting NVSNAP_CUDA_PARALLELISM precisely because its
effective value was not visible anywhere. A configuration surface you cannot read
is one you end up guessing about.
The most significant example is NVSNAP_CRIU_V2
(internal/agent/checkpoint_v2.go:85): which capture engine runs is decided by an
environment variable with no chart surface, no schema and no validation.
A recent change of ours added a 41st and it has already been removed
(NVSNAP_DUMP_PIDNS_ROOT, commit 625f556ac) rather than merged, on the grounds
that an env switch which changes what a capture contains is invisible, untyped
and outlives the experiment that introduced it. That reasoning applies more
broadly.
Proposed categories
Not every one of these is wrong. They should be sorted before anything is
changed:
-
Process-to-process wiring, legitimate as env. The agent or webhook sets these
for a child it launches; they are not user configuration.
NVSNAP_ORIG_COMMAND,NVSNAP_ORIG_CWD,NVSNAP_CHECKPOINT_HOST_PATH,
NVSNAP_CAPTURE_NODE,NVSNAP_AGENT_TOKEN,NVSNAP_VERSION. -
Behaviour switches that change what the system does. These should be agent
flags plumbed through chart values, with the value visible in the rendered
spec.
NVSNAP_CRIU_V2,NVSNAP_DEFAULT_CAPTURE_PATH,NVSNAP_CAPTURE_PLAN,
NVSNAP_NO_OVERLAY,NVSNAP_LAZY_PAGES,NVSNAP_COMPRESS_CHECKPOINT,
NVSNAP_CRIU_V2_COMPRESS,NVSNAP_REPLAY_MOUNTS,NVSNAP_ALLOW_WHOLE_ROOTFS. -
Tuning knobs. Chart values with documented defaults and range validation.
NVSNAP_CUDA_PARALLELISM,NVSNAP_PEER_FETCH_CONCURRENCY,
NVSNAP_PREP_DEADLINE,NVSNAP_PREP_POLL,NVSNAP_KEEPALIVE_SECONDS. -
Debug-only. Keep as env, but name them consistently and confirm they are inert
by default.
NVSNAP_STRACE_ENABLED,NVSNAP_DEBUG_IO_URING,NVSNAP_DISABLE_IO_URING_REINIT. -
Deployment wiring that belongs in values.
NVSNAP_L2_*,NVSNAP_REPLICATION_*,NVSNAP_BLOB_STORE_URL,
NVSNAP_CATALOG_URL,NVSNAP_WEBHOOK_AGENT_BASE_URL,NVSNAP_FSSTORE_PATH,
NVSNAP_POD_CACHE_DIR,NVSNAP_WORKLOADS_DIR. -
Dead. Some of these are likely unused; delete rather than migrate.
Tasks
- Complete the inventory, including any non-
NVSNAP_prefixed reads. - Sort every variable into the categories above and confirm which are dead.
- Migrate category 2 and 3 to agent flags plus chart values, keeping the env
var as a deprecated fallback for one release. - Log effective configuration once at agent startup, so a running cluster can
be inspected from logs rather than by exec. - Fail startup on an unrecognised
NVSNAP_*variable, so a typo is loud
rather than silently ignored. - Document the surviving set.
Notes
Reproduce the inventory with:
grep -rnoE 'os\.Getenv\("NVSNAP_[A-Z0-9_]+"\)' --include=*.go internal/ cmd/
The startup-logging and unknown-variable tasks are worth doing first even if the
migration is slow: both make the current state visible without changing any
behaviour.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.