rust-lang / rust-lang/rustup

Rustup unnecessarily re-installs all components if cached manifest changes

Open
#3,027 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Rust
Stars
7k
Forks
1.1k
Avg merge
22h 40m
Merged PRs (30d)
46

Description

Problem

If a channel manifest is updated but a component maintains the same checksum, even within a single Rust version, rustup will download and install all components for that toolchain again.

Steps
  1. Run a proxy server on localhost that changes a fetched channel manifest's date and the corresponding .sha256 file, but nothing else.
  2. rustup update
  3. env RUSTUP_DIST_SERVER=localhost rustup update

The second rustup update ends up re-downloading and re-installing every component of the toolchain, even though none of the hashes for those components have changed (< and > lines are emitted by the proxy):

info: syncing channel updates for 'stable-aarch64-unknown-linux-gnu'
< GET /dist/channel-rust-stable.toml.sha256
> channel-rust-stable.toml.sha256
< GET /dist/channel-rust-stable.toml
> channel-rust-stable.toml
< GET /dist/channel-rust-stable.toml.asc
> channel-rust-stable.toml.asc
warning: Signature verification failed for 'http://127.0.0.1:38537/dist/channel-rust-stable.toml'
info: latest update on 2022-06-31, rust version 1.62.0 (a8314ef7d 2022-06-27)
info: downloading component 'cargo'
< GET /dist/2022-06-30/cargo-1.62.0-aarch64-unknown-linux-gnu.tar.xz
> cargo-1.62.0-aarch64-unknown-linux-gnu.tar.xz
info: downloading component 'rust-std'
< GET /dist/2022-06-30/rust-std-1.62.0-aarch64-unknown-linux-gnu.tar.xz
> rust-std-1.62.0-aarch64-unknown-linux-gnu.tar.xz
info: downloading component 'rustc'
< GET /dist/2022-06-30/rustc-1.62.0-aarch64-unknown-linux-gnu.tar.xz
> rustc-1.62.0-aarch64-unknown-linux-gnu.tar.xz
info: removing previous version of component 'cargo'
info: removing previous version of component 'rust-std'
info: removing previous version of component 'rustc'
info: installing component 'cargo'
info: installing component 'rust-std'
 25.4 MiB /  25.4 MiB (100 %)  13.6 MiB/s in  1s ETA:  0s
info: installing component 'rustc'
 48.2 MiB /  48.2 MiB (100 %)  14.7 MiB/s in  3s ETA:  0s

  stable-aarch64-unknown-linux-gnu updated - rustc 1.62.0 (from rustc 1.62.0)

info: cleaning up downloads & tmp directories
Possible Solution(s)

Rustup should probably realize that components whose checksums haven't changed from the existing install do not need to be re-fetched/installed.

Notes

No response

Rustup version
rustup 1.24.3 (ce5817a94 2021-05-31)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: syncing channel updates for 'stable-x86_64-apple-darwin'
info: The currently active `rustc` version is `rustc 1.62.0 (a8314ef7d 2022-06-27)`
Installed toolchains
Default host: aarch64-unknown-linux-gnu
rustup home:  /home/jongje/.rustup

installed toolchains
--------------------

stable-aarch64-unknown-linux-gnu (default)
nightly-aarch64-unknown-linux-gnu
1.60.0-aarch64-unknown-linux-gnu
1.61.0-aarch64-unknown-linux-gnu

active toolchain
----------------

stable-aarch64-unknown-linux-gnu (default)
rustc 1.62.0 (a8314ef7d 2022-06-27)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the rustup update flow and reproduce the issue using the localhost proxy described in the steps. Compare the component checksums across the two manifests and trace why unchanged components are downloaded, removed, and installed again. Done means an unchanged component is neither re-fetched nor re-installed when only the manifest date changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.