Proposal: special toolchain names referring to particular override levels

Part of https://github.com/rust-lang/rustup/issues/4059.

Since https://github.com/rust-lang/rustup/pull/4932 it has been decided that only characters satisfying GeneralSecurityProfile::identifier_allowed() should be allowed in rustup toolchain names.

This has opened up new possibilities for us to assemble our special/reserved toolchain names as needed with special characters such as @ or =.

Following up the above efforts, noting the five levels of rustup toolchain overrides:

1. The +<toolchain> override.
2. The RUSTUP_TOOLCHAIN environment variable.
3. The rustup override set override.
4. The rust-toolchain.toml override.
5. The default toolchain.

... aiming to address https://github.com/rust-lang/rustup/issues/4391, I propose the following changes:

• Make env and default new reserved words in addition to the existing none.
• Introduce the following toolchain syntax:
  • env: Refers to the toolchain that should be activated by the env override.
    • This will not be useful per se but sometime people want to be explicit, e.g. rustup run env cargo instead of cargo.
  • dir=<dir>: Refers to the toolchain that should be activated by the dir override on <dir> or its fallback dirs.
    • e.g. cargo +dir=./crates/download
  • file=<file>: Refers to the toolchain that should be activated by the TOML override in <file> which follows the rust-toolchain.toml schema.
    • e.g. cargo +file=windows.toml, for a Windows-specific rust-toolchain.toml setup.
  • default: Refers to the default toolchain. Bails out if none is set.
    • e.g. cargo +default

Concerns

• How should we handle the error for dir= and file= when the dir/TOML override doesn't exist? Should we bail out or warn and fall back to the next level?
  • My guess is that we should warn and fall back.
• Consider the relation between this proposal and #4945. Should we resolve the toolchain name when the user issues rustup default file=windows.toml? If so, should we fully resolve or partially resolve it?
  • My guess is that we should partially resolve to an official or a custom toolchain name (that isn't a reserved word and doesn't have special characters like =).
• Can we use this in RUSTUP_TOOLCHAIN (is RUSTUP_TOOLCHAIN not guaranteed to be a parsable toolchain name by downstreams other than rustup)? If we can, we will also resolve #2793 at the same time by setting RUST_TOOLCHAIN=default.
  • I think we probably can.

Note

This also simplifies https://github.com/rust-lang/cargo/issues/10098 because this involves changing the override lookup logic to work with arbitrary base paths. In that case, we'd just need to alter the path for our dir/TOML override source by prefixing it with the path passed via cargo -C.