rust-lang / rust-lang/cargo

`cargo tree` doesn't handle transitive duplications properly

Open
#9,599 7 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

C-bug Command-tree E-medium S-needs-design
Dominant language
Rust
Stars
15.5k
Forks
3k
Avg merge
23h 30m
Merged PRs (30d)
51

Description

Problem
There are some situations where cargo tree will mark a package as a duplicate (*) when it probably shouldn't (and shows the wrong features with --no-dedupe). This arises when a package is built twice with the same features, but with different dependencies.

A real-world example is when looking at the following:

[package]
name = "foo"
version = "0.1.0"
resolver = "2"

[dependencies]
diesel = { version = "=1.4.7", features = ["postgres"] }
diesel_migrations = "=1.4.0"

Running cargo tree -f '{p} {f}' results in:

~/Proj/rust/cargo/scratch/diesel-issue> cargo tree -f '{p} {f}'
diesel-issue v0.1.0 (/Users/eric/Proj/rust/cargo/scratch/diesel-issue)
├── diesel v1.4.7 32-column-tables,bitflags,default,postgres,pq-sys,with-deprecated
│   ├── bitflags v1.2.1 default
│   ├── byteorder v1.4.3 default,std
│   ├── diesel_derives v1.4.1 (proc-macro) default,postgres
│   │   ├── proc-macro2 v1.0.27 default,proc-macro
│   │   │   └── unicode-xid v0.2.2 default
│   │   ├── quote v1.0.9 default,proc-macro
│   │   │   └── proc-macro2 v1.0.27 default,proc-macro (*)
│   │   └── syn v1.0.73 clone-impls,default,derive,extra-traits,fold,full,parsing,printing,proc-macro,quote
│   │       ├── proc-macro2 v1.0.27 default,proc-macro (*)
│   │       ├── quote v1.0.9 default,proc-macro (*)
│   │       └── unicode-xid v0.2.2 default
│   └── pq-sys v0.4.6
└── diesel_migrations v1.4.0 default
    ├── migrations_internals v1.4.1 default
    │   └── diesel v1.4.7 32-column-tables,bitflags,default,postgres,pq-sys,with-deprecated (*)
    └── migrations_macros v1.4.2 (proc-macro) default
        ├── migrations_internals v1.4.1 default (*)    <-- PROBLEM IS HERE
        ├── proc-macro2 v1.0.27 default,proc-macro (*)
        ├── quote v1.0.9 default,proc-macro (*)
        └── syn v1.0.73 clone-impls,default,derive,extra-traits,fold,full,parsing,printing,proc-macro,quote (*)

The problem is the diesel-issue → diesel_migrations → migrations_macros → migrations_internals. It has a (*) to indicate that it is duplicated. However, migrations_internals is built twice, and this is deceiving because the second migrations_internals has a dependency on diesel with no features.

Running with --no-dedupe is even worse, because it shows the wrong features for diesel under migrations_internals.

Steps
The following is an example in Cargo's testsuite format:

#[cargo_test]
fn dedupe_transitive_features2() {
    Package::new("differs", "1.0.0")
        .feature("some-feat", &[])
        .publish();
    Package::new("shared", "1.0.0")
        .dep("differs", "1.0")
        .publish();

    let p = project()
        .file(
            "Cargo.toml",
            r#"
                [package]
                name = "foo"
                version = "0.1.0"
                resolver = "2"

                [dependencies]
                shared = "1.0"
                differs = {version="1.0", features=["some-feat"]}
                bar = {path="bar"}

                [build-dependencies]
            "#,
        )
        .file("src/lib.rs", "")
        .file(
            "bar/Cargo.toml",
            r#"
                [package]
                name = "bar"
                version = "0.1.0"

                [build-dependencies]
                shared = "1.0"
            "#,
        )
        .file("bar/src/lib.rs", "")
        .build();

    // ISSUE HERE: The last `shared` shouldn't be `(*)`
    p.cargo("tree -f")
        .arg("{p} [{f}]")
        .with_stdout(
            "\
foo v0.1.0 [..]
├── bar v0.1.0 [..]
│   [build-dependencies]
│   └── shared v1.0.0 []
│       └── differs v1.0.0 []
├── differs v1.0.0 [some-feat]
└── shared v1.0.0 [] (*)
",
        )
        .run();
}

    // ISSUE HERE: The last `differs` shows with no features!
    p.cargo("tree --no-dedupe -f")
        .arg("{p} [{f}]")
        .with_stdout(
            "\
foo v0.1.0 [..]
├── bar v0.1.0 [..]
│   [build-dependencies]
│   └── shared v1.0.0 []
│       └── differs v1.0.0 []
├── differs v1.0.0 [some-feat]
└── shared v1.0.0 []
    └── differs v1.0.0 [some-feat]
",
        )
        .run();

Possible Solution(s)
The issue is that cargo tree doesn't have the same logic that was added in #8701 to accommodate dependencies that are the same, but link to different dependencies.

I have toyed with the idea of changing cargo tree to use the UnitGraph computed for a normal build instead of trying to recreate how some of these things are computed. There are some complexities and downsides to that approach, but I continue to feel that trying to replicate this logic in multiple places is not a good idea.

Notes

Output of cargo version: cargo 1.54.0-nightly (44456677b 2021-06-12)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the cargo testsuite case dedupe_transitive_features2 and run both cargo tree -f and cargo tree --no-dedupe -f using its expected output. Trace how cargo tree handles transitive dependencies with different features, especially the relationship to the UnitGraph logic from #8701. Done means the final shared dependency is not incorrectly marked (*), and --no-dedupe reports the correct features.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
build-system, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.