Pinning transitive dependencies with a lockfile

• Feature Request.

Configuration

Operating system: Windows 10

PlatformIO Version (platformio --version): 6.1.6

Description of problem

It’s good practice to pin your dependencies to exact versions to avoid unexpected regressions, but transitive dependency updates can also cause unexpected regressions, and pinning your transitive dependencies is currently painful.

You can check in your project’s .pio directory to version control, which has the added benefit of protecting against packages being changed or removed, but it only includes your lib_deps and not your platform_packages. You can add everything in your pio pkg list tree to your platform_packages and lib_deps, but you have to be careful to put the deepest dependencies first (platformio/platformio-core#2115), and mixing direct dependencies with transitive dependencies makes updating hard.

Many libraries don’t constrain their dependencies at all. Even if all the libraries you use constrain their own dependencies to those that are semver compatible, the only way to avoid breakage entirely (and have reproducible builds) without pinning your transitive dependencies is to only depend on libraries that pin everything to exact versions, which is impossible in practice.

Having a lockfile like Rust’s Cargo.lock or npm’s package-lock.json that lists all of your direct and transitive dependencies, with their exact version numbers and hashes of their contents, would fix most of these problems. When installing packages for a project, we would use the lockfile in the top-level project if any, and ignore any lockfiles in dependencies.

It may not prevent packages being removed though, because even packages on the PlatformIO Registry are fetched from their original repos, but the only way to avoid that is to vendor your entire dependency tree (copy everything into your project).

No repository files or tests are named. Start by tracing the existing pio pkg list, lib_deps, and platform_packages dependency-resolution paths, then compare the requested behavior with Cargo.lock and package-lock.json. Done means the project can use a top-level lockfile to record exact direct and transitive package versions and content hashes during installation.