[RFC] Package subsetting support for vendored code

Desired Behavior

When other Dune projects are vendored, I would like to be able to define which packages are visible to Dune. This is currently supported using the -p flag to dune build when building/installing with OPAM: in that case Dune only builds (and OPAM installs) only the artifacts that are part of the packages specified via the -p flag.

This allows to use multiple third party libraries that contain the same package name (e.g. dune packages that dune vendors) or multiple versions of the same package (e.g. libfoo-1.0 and libbar-1.0.1 from the same dev-repo, sharing a dune-project but installed in different versions).

Unfortunately, the same is not possible when vendoring code, as in such case dune aborts because the same packages exist multiple times and it doesn't know which code to pick.

Example

Currently to designate a directory as vendored we can use

(vendored_dirs *)

In this RFC I would like to suggest an extension to the vendored_dirs stanza that would allow specifying additional metadata on the directories to be vendored:

(vendored_dirs 
  (foo (packages foo))
  (bar (packages bar))
  *)

Would in such case use the directory name foo and only build the packages that are foo, bar would do similarly, while the other directories would have all their packages evaluated.

Other solutions

I've attempted to write a tool to rename packages to random names (e.g. package bar in dir foo would become bar-ab43ec and foo in dir bar would become foo-e21fce, but unfortunately such a tool is not possible without running into a lot of problems:

• A lot of dune files need to be rewritten to patch the project-internal references, despite ending up not actually building the renamed packages.
• dune-project files need to be rewritten to rename the package stanzas
• <project>.opam files need to be renamed

While these issues are solvable with a bit of elbow-grease, the thing that broke the camels back was

• dune files need to have their (package) stanzas patched in rules
• This is only possible in dune files that exist statically. If the rules are generated, e.g. using an ML file either as dune file or as generator that generates dune.inc-like files that need to contain references to packages, these generators would need to be patched which is not possible.

As such, my suggestion is to implement this extension as part of Dune, which has all the information to subselect packages in a dune project already (with -p) so the extension is most likely rather limited in scope.

Start by reading the existing vendored_dirs handling and the package-selection behavior behind dune build -p. Compare how vendored directories and OPAM package selection are represented, then define the metadata and behavior needed for selecting packages per directory. Done means the RFC is resolved into an agreed implementation scope, including validation and tests for duplicate vendored packages.