[Linux][Install] uninstall cannot remove Hugging Face model cache because vLLM downloads it as root inside the container
- Dominant language
- TypeScript
- Stars
- 22.5k
- Forks
- 3.1k
- Avg merge
- 1d 43m
- Merged PRs (30d)
- 718
Description
## Description
`nemoclaw uninstall --delete-models --destroy-user-data` prints an explicit failure while trying to clean the Hugging Face model cache: "Hugging Face cache-data cleanup failed: Hugging Face model cache is not current-user filesystem authority". This exact string appeared on 8 separate host/run combinations across 5 distinct machines in one test campaign - every Linux host whose provider ever exercises the HF-cache-backed managed vLLM path. On a 6th machine, a manual `rm -rf ~/.cache/huggingface` (plus a passwordless-sudo fallback) also hit literal "Permission denied" on specific blob files under the current user's own cache directory - independent corroborating evidence that specific files are owned by a different user, not that the parent directory lacks permissions.
Root cause: the vLLM container bind-mounts the host's Hugging Face cache directory into the container at `/root/.cache/huggingface`, and the container is not run as a non-root user. Any model weights the container downloads therefore land on the host filesystem owned by root - which the host's regular user (and `nemoclaw uninstall`, which does not attempt privilege elevation for this cleanup) can never remove afterward.
**Platform scope:** Reproduced across 6 distinct Linux machines (DGX Station, DGX Spark, Ubuntu, WSL2/ARM) in one campaign, all confirmed genuine v0.0.116 via each host's own Resolved install ref banner. Not tested on macOS (vLLM's managed-container path is Linux-only, there is no equivalent local-vLLM flow on macOS).
**Regression:** Unknown - earlier versions not independently bisected. A related feature request (#8917, "Make uninstall --delete-models remove all installed host models") was fixed by a PR that added Hugging Face cache deletion to the `--delete-models` flag in the first place (confirmed merged and present in v0.0.116) - that PR is what introduced the code path this bug now fails inside; it did not itself create the root-ownership problem (vLLM's container has always run as root), it only added the (currently-failing) attempt to clean up after it.
## Environment
```text
Device: DGX Station / DGX Spark / Ubuntu server / WSL2 ARM (multiple Linux hosts)
OS: Ubuntu 24.04 LTS (representative; also reproduced on WSL2)
Architecture: aarch64 and x86_64 (both affected)
Node.js: Not captured in source evidence
npm: Not captured in source evidence
Docker: Not captured in source evidence
OpenShell CLI: Not captured in source evidence
NemoClaw: v0.0.116 (confirmed via Resolved install ref banner on all 6 hosts)
OpenClaw: N/A (this is a post-onboard uninstall-path defect, not tied to a specific agent)
```
## Steps to Reproduce
1. On a Linux host, onboard using a provider that downloads a model into the managed local vLLM Hugging Face cache path (e.g. Express install, or `NEMOCLAW_PROVIDER=install-vllm`).
2. Let onboarding complete successfully (or reach whatever state actually downloads model weights).
3. Run:
```bash
nemoclaw uninstall --yes --delete-models --destroy-user-data
```
## Expected Result
Uninstall with `--delete-models` successfully removes the downloaded Hugging Face model cache, matching the flag's own stated purpose of deleting all installed host models.
## Actual Result
```text
[5/6] Model stores
No installed Ollama models found.
Hugging Face cache-data cleanup failed: Hugging Face model cache is not current-user filesystem authority
Hugging Face cache-data cleanup did not complete during Model stores. Resolve the reported ownership or path error and retry uninstall.
[6/6] State and binaries
...
Uninstall completed with errors. Some state may remain on disk; see warnings above.
```
Identical error text observed on 8 separate host/run combinations across 5 machines. On a 6th machine, manually running `rm -rf ~/.cache/huggingface` as the same user that performed the onboard hit:
```text
rm: cannot remove '.../models--nvidia--Qwen3.6-35B-A3B-NVFP4/blobs/07141c2db92e47bc08777132cd1a0323faf300eab3a7d7c111bc2bf075fda050': Permission denied
(and four more blob files with the same error)
```
A passwordless-sudo fallback attempt after the plain `rm` also did not recover these files.
## Logs
```text
Representative uninstall output (one host):
[5/6] Model stores
No installed Ollama models found.
Hugging Face cache-data cleanup failed: Hugging Face model cache is not current-user filesystem authority
Hugging Face cache-data cleanup did not complete during Model stores. Resolve the reported ownership or path error and retry uninstall.
Representative manual-cleanup output (a different host), confirming root ownership via Permission denied on specific blob files under the current user's own cache directory:
rm: cannot remove '.../models--nvidia--Qwen3.6-35B-A3B-NVFP4/blobs/07141c2db92e47bc08777132cd1a0323faf300eab3a7d7c111bc2bf075fda050': Permission denied
rm: cannot remove '.../models--nvidia--Qwen3.6-35B-A3B-NVFP4/blobs/0aa0ce0658d60ac4a5d609f4eadb0e8e43514176': Permission denied
rm: cannot remove '.../models--nvidia--Qwen3.6-35B-A3B-NVFP4/blobs/023756cfadf88e5bf69eefeee3e172f38c448d64': Permission denied
```
## Related
#8917 (fixed, confirmed present in v0.0.116) added Hugging Face cache deletion to the `--delete-models` uninstall flag. This report is a new defect in that same code path: the deletion attempt itself fails because of a pre-existing root-ownership condition the feature did not account for.
#7088 (fixed, confirmed present in v0.0.116) reported the same underlying root cause - NemoClaw's managed vLLM onboard container runs as root and plants root-owned files into the host-mounted Hugging Face cache - but for a different code path: that report was about the onboard preflight check aborting a subsequent onboard/model-switch when it walked into root-owned leftovers. This report is a different code path, uninstall's own `--delete-models` cleanup step failing to remove those same root-owned files. The #7088 fix appears to have addressed onboard's preflight check; it does not appear to have removed the underlying root-ownership condition, since `--delete-models` still cannot clean it up here.
Contributor guide
Assessment
This issue has not been assessed yet.