User-defined flags for compile-time behaviour

Over in the mina project we've been using the dune profile to choose the compile-time configuration to build with, with the help of a fairly unpleasant hack. This works well, in the sense that we can build with the chosen configuration, but also has a few limitations:

• changing the profile invalidates all build assets, even those that do not depend on the compile-time configuration, so switching between configurations results in a complete recompile
• there is no fine-grained control over the options in the configuration.

To address these limitations, it would be helpful to add the ability to pass custom flags to dune, for example --custom-flag foo=bar. The most ideal implementation then might have

• %{custom_flag:foo} variables
  • this would be the most powerful if it can also be used in (library (enabled_if ...)) stanzas, but appearing in normal variable positions would already be a big improvement
• a ppx cookie exposing the custom flags
  • this would be handy for a ppx_optcomp-style preprocessor
  • this would presumably also need some way to declare these flags as preprocessor dependencies, and perhaps also to limit the available flags to those listed as dependencies.

I imagine that the hardest part of this will be invalidating the correct build artifacts when changing flags, but hopefully something similar to what's already done with environment variables will work.

I'm happy to work on a prototype for this, if it seems like a reasonable feature to add.

Start by reading Dune's existing profile and environment-variable handling, then compare the proposed command-line custom flags with the %{custom_flag:foo} and ppx cookie requirements described here. Done means defining the feature's scope and invalidation behavior, including whether flags work in enabled_if stanzas and how preprocessor dependencies are declared.