Future of vendoring in Dune

This is a ticket to centralise the discussion about the future of vendoring in Dune.

It is now clear to us that the current semantic of vendoring in Dune is not very good and has a lot of rough edges, on which users regularly cut themselves. We started an effort to clarify the semantic of vendoring in Dune and are working towards aligning the implementation with this new semantic.

Moving forward, we should link tickets/discussions about problem related to Dune and vendoring to this issue. Users are also encouraged to report issues they are experiencing with vendoring here so that we can reason about whether the new semantic will help.

New semantic

The new semantic we want to enforce is as follow: vendoring a project V into a scope X means demoting all the public elements of V as private elements of X.

For instance, public packages names, public binaries or public libraries of V will be visible inside X, but not in the rest of the workspace. Just like private libraries are only visible in the project where they are defined.

To clarify what we mean by a scope here: it is the subtree rooted at the enclosing project root. So for instance, with the following directory structure:

- dune-project
- dune
- x/dune-project
- x/dune
- x/y/dune-project
- x/y/dune
- x/vendor/dune-project (with x/vendor marked as vendored in x/dune)
- x/vendor/dune
- z/dune-project
- z/dune

Public elements defined in x/vendor/dune are visible in x/dune and x/y/dune but not in dune or z/dune.

Start with the vendoring semantics described in this issue and inspect the referenced dune-project and dune files in the example directory tree. Determine how vendored public packages, binaries, and libraries are currently scoped; done means the implementation consistently enforces the proposed subtree visibility rules, though this issue does not name tests or specific source files.