ROCm / ROCm/rocm-cli

uv cache does not follow --prefix, so prefixed installs on another filesystem still copy instead of hardlink

Open
#187 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
40
Forks
9
Avg merge
4d 20h
Merged PRs (30d)
59

Description

Problem

#170 points uv at a cache under the managed data directory so it shares a filesystem with the environments uv populates and hardlinking works. That covers the default and ROCM_CLI_DATA_DIR cases, but not --prefix, which is the mainline flow — the built-in assistant's system prompt instructs the model to always obtain an install folder and pass --prefix, and it cascades to ComfyUI, which roots its app at install_root/apps/<id>.

--prefix relocates install_root for one SDK install and leaves paths.data_dir untouched. The two are never cross-checked, so with ROCM_CLI_DATA_DIR and --prefix on different filesystems, uv falls back to copying every file and each environment carries a full duplicate of the ROCm SDK and torch stack.

This also contradicts the documented convention in docs/testing.md and docs/manual-testing.md, which specify a localized cache inside the selected ROCm folder at <install-root>/pip-cache, explicitly including --prefix folders.

Why it was not fixed in #170

The obvious fix — thread the target root through and use <install_root>/uv-cachedoes not work, and breaks every wheel install. uv materializes UV_CACHE_DIR on every invocation, including uv venv, and uv venv refuses to create an environment in a directory that already exists.

Verified with uv 0.9.x:

$ UV_CACHE_DIR=$P/fresh/uv-cache uv venv --python 3.12 $P/fresh
Using CPython 3.12.3 interpreter at: /usr/bin/python3.12
Creating virtual environment at: /tmp/.../fresh
error: Failed to create virtual environment
  Caused by: A directory already exists at: /tmp/.../fresh

$ ls -la $P/fresh
drwxr-xr-x  uv-cache      <-- uv created the cache dir, which then blocked venv creation

Because install_root is the venv directory (venv_python_path(&install_root)), the cache cannot live inside it. This is also why the existing pip-cache convention worked: pip only creates its cache during downloads, which happen after the venv exists, whereas uv both creates the venv and does the downloading.

So this is not a signature change — it needs a cache-location scheme decided first.

Options for maintainers

  1. Sibling of the install root (<install_root>-uv-cache, or <install_root>/../uv-cache). Same filesystem, no collision with uv venv. Pollutes the user's chosen directory or its parent, and for --prefix $HOME/envs means $HOME/uv-cache.
  2. Cache only the heavy phase against the install root — keep uv venv on the data-dir cache and switch to an install-root cache for uv pip install once the venv exists. Works, but splits the cache across two locations and is fragile against reordering.
  3. Validate instead of relocate — detect at install time that --prefix and the cache are on different filesystems and warn, pointing at ROCM_CLI_UV_CACHE_DIR.
  4. Keep one cache and accept the copy for cross-filesystem --prefix.

Related

  • The Downloads/cache: <value> line rendered from active_runtime_pip_cache_dir / setup_runtime_pip_cache_dir falls back to <install_root>/pip-cache, but nothing sets PIP_CACHE_DIR or --cache-dir and every manifest constructor writes pip_cache_dir: None. The CLI therefore tells the user downloads stay in their chosen ROCm folder while they actually go to the uv cache. Pre-existing; resolving this issue is the natural place to make that message true.

Current state

#170 documents the gap in docs/manual-testing.md, pins it with uv_cache_does_not_follow_a_prefix_install_root, and offers ROCM_CLI_UV_CACHE_DIR as a manual workaround.

Follow-up to #170.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the prefix handling around paths.data_dir and the uv invocation, then read docs/testing.md, docs/manual-testing.md, and the regression test uv_cache_does_not_follow_a_prefix_install_root from #170. Compare the listed cache-location options and verify that the chosen design avoids uv venv directory collisions, preserves same-filesystem hardlinking, and makes the displayed cache location accurate.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.