Pluggable output backends

I was chatting with @pvdrz at the Rust for Linux conference about some bold ideas for bindgen and he thought I should write them up, so here goes.

The problem space

cxx is a pretty popular C++ interop tool these days. Its biggest advantage is that Rust code manipulates C++ data using friendly vocabulary types such as cxx::UniquePtr<T> and cxx::CxxString. These make FFI much safer in practice than bindgen-style bindings, because for example there's no need to track object lifetimes, nor string lengths, etc. Irrespective of the specifics of the policy relating to the unsafe keyword, cxx-style interop is much less error prone in practice than bindgen-style FFI.

However, cxx can't generate such bindings from pre-existing C++ headers. The full language boundary needs to be declared in an IDL (which looks a lot like Rust code but actually isn't quite).

For years now, the Chromium project has wanted to combine the ergonomic advantages of cxx with bindgen-style generation from headers:

• Our manifesto from 2020, which is now a little out of date
• autocxx, a project which literally connects bindgen into cxx
• crubit, a similar concept not reliant on bindgen nor cxx

The idea

The idea is: bindgen can output different "flavors" of bindings. Perhaps this is pluggable, perhaps there are multiple flavors built in.

• Basic flavor: current bindgen output. Lots of pointers.
• cxx-like flavor: detects things like std::unique_ptr and substitutes them with cxx::UniquePtr, generating both C++ and Rust side code to make this possible. This is basically what autocxx does but it's really pretty complicated since it's a post-processor, a lot of the complexity would drop away if it were built into bindgen itself.
• Future flavors could use a CppRef<T> type for C++ references, etc.

What this would require

It would require:

• bindgen gains the ability to generate C++ side shim functions as well as Rust side code.
• Some amount of modularity so that these output backends are not very invasive into the bindgen code. To give an indication of what this takes, autocxx has had to fork bindgen primarily to add extra information about the generated items. The fork is here, with a partial list of the reasons here.

Simplest first step

All this sounds vague. A hypothetical first step could be a --emit-cxx-crate-strings which:

• Spots all use of const &std::string. If a C++ function takes such a parameter,
• A synthetic Rust-side function is created which takes a &cxx::CxxString and then calls the real function

This is just a small step, but users of both cxx and bindgen could immediately benefit because they can create such strings in Rust.

Simplest second step

The next step after that would be to support std::string by value, which would require either cunning to store self-referential types on the Rust stack or, more likely, wrapping the std::string in a std::unique_ptr. This would require a small C++ shim function to be generated to unwrap it then pass it into the original C++ API.

The issue names no source files or tests; begin by tracing bindgen's existing output-generation path and considering the proposed --emit-cxx-crate-strings entry point. Done would require an agreed, scoped backend design with a concrete first implementation step and clear behavior for generated Rust and C++ code.