Add API to call `codeview_annotation` intrinsic

Proposal

Problem statement

At Microsoft we are working on supporting Rust drivers for Windows. To implement tracing in drivers we need a way to write arbitrary UTF-8 strings to the PDB at compile time.

Suppose the user writes this trace statement in the driver:

trace!("Bytes {}, duration {} ms", byte_count, elapsed_ms);

then we want the fixed metadata like the format string "Bytes {}, duration {} ms" and the types of variables byte_count and elapsed_ms to be written to the PDB while the runtime values of these variables are emitted to Windows' tracing infrastructure.

Later, tooling can read the runtime values from tracing infra, combine them with the metadata from the PDB and produce a human readable log.

This allows tracing to work efficiently by not having to emit the fixed metadata in every invocation of the trace statement.

We have already opened an MCP to add a compiler intrinsic for writing strings to the PDB. We now need a public API for calling that intrinsic.

[!NOTE]
trace!() is a macro owned by us and is not part of this proposal.

Motivating examples or use cases

This repo contains an example driver showing how we intend to implement tracing using the intrinsic and the wrapper API. I suggest going over the source code of the driver in lib.rs first and then seeing the expansion in the README.

Outside of our use-case, such an API can be useful in general for stashing away metadata in a PDB that can be consumed during debugging or at run time.

Solution sketch

The proposed API is as follows:

#[unstable(feature = "codeview_annotation", issue = "...")]
pub trait CodeViewAnnotationArgs {
    const ARGS: &[&str]; // String args to be written to the PDB
}

#[inline(always)]
#[unstable(feature = "codeview_annotation", issue = "...")]
pub fn codeview_annotation<T: CodeViewAnnotationArgs>() {
    crate::intrinsics::codeview_annotation::<T>();
}

The API simply forwards to the intrinsic which has the same name and signature.

CodeViewAnnotationArgs::ARGS carries the input strings to be written to the PDB. Taking them as an associated const on a trait instead of ordinary function params ensures they are available at compile time as required by the underlying intrinsic.

See the accompanying PR for more details of both the API and the intrinsic.

Lowering

The Rust intrinsic lowers to a call to llvm.codeview.annotation LLVM intrinsic. The LLVM intrinsic writes the strings to the PDB in the form of an S_ANNOTATION record.

Arguments

The arguments must be const-evaluable &str values backed by literals, named constants, associated constants and supported statics.

Empty strings (e.g. &["", ""]) and empty list of strings (e.g. &[]) are both supported. The empty list of strings just results in an empty S_ANNOTATION record being emitted in the PDB.

Location

The API will be located under core::hint because it has no runtime effect.

Failure

If CodeViewAnnotationArgs::ARGS fails to const evaluate, compilation will fail with a standard const evaluation diagnostic.

Usage

To call the API users will need to:

• Declare a type (say Args) implementing CodeViewAnnotationArgs. The type can be generic or non-generic
• Set CodeViewAnnotationArgs::ARGS to the string args
• Invoke the API with that type e.g.: codeview_annotation::<Args>()

Here are some example invocations with different kinds of strings:

Literals and Consts

const WORLD: &str = "world";

struct Args;

impl CodeViewAnnotationArgs for Args {
    const ARGS: &[&str] = &["hello", WORLD];
}

codeview_annotation::<Args>();

Associated Consts

trait GetName {
    const NAME: &str;
}

struct Field;

impl GetName for Field {
    const NAME: &str = "Foo";
}

struct Args;

impl CodeViewAnnotationArgs for Args {
    const ARGS: &[&str] = &["metadata", Field::NAME];
}

codeview_annotation::<Args>();

Associated Consts on Generic Types

This example shows how the user can start with some variables in their code (a and b), infer their types and then emit strings associated with their types as annotations.

// A trait that lets you associate a
// string `NAME` with any type
trait GetName {
    const NAME: &str;
}

// The struct `Args`, its impl of `CodeViewAnnotationArgs`
// and the `emit_annotation` wrapper function work together to
// invoke `codeview_annotation` with the `NAME` associated
// with the types of args `_a` and `_b`
struct Args<A, B>(std::marker::PhantomData<(A, B)>);
impl<A: GetName, B: GetName> CodeViewAnnotationArgs for Args<A, B> {
    const ARGS: &[&str] = &["metadata", A::NAME, B::NAME];
}
fn emit_annotation<A: GetName, B: GetName>(_a: &A, _b: &B) {
    codeview_annotation::<Args<A, B>>();
}

// This is how `codeview_annotation` is eventually invoked
// given some variables `a` and `b`
emit_annotation(&a, &b);

[!NOTE]
This example is closer to how we actually intend to use the intrinsic in a driver.

Supported platforms

The underlying intrinsic supports only:

• Targets with PDB debug info and
• The LLVM backend

and is a no-op on all the other targets and backends.

The same holds true for the API.

Alternatives

The following alternatives to the API and the intrinsic were considered but rejected.

Alternatives to the proposed API

• Const generics-based signature e.g. codeview_annotation<const ARGS: &[&str]>(). However, that does not work because const generics currently do not support unsized types. We would definitely prefer this signature if the unsized type support becomes available, but as per @RalfJung it is a hard problem and may never happen.
• A macro instead of a function. It does not work either because a macro does not offer an ergonomic way of enforcing const-ness of string args.

Alternatives to the intrinsic

• Using the driver's binary instead of the PDB to carry the metadata strings. That proved to be brittle and complex and it exposed proprietary implementation details which is not acceptable for many Windows drivers.
• Using #[link_name] to get the strings into the PDB. Is awkward to use, limited to fixed string values and does not work with our tooling without extensive changes (there is a wide variety of tools spread across many teams).

See the Zulip discussion associated with the MCP for more details.

Can it be done in a crate?

As per convention, public APIs wrapping compiler intrinsics live in the std library so a crate is not a good fit.

Links and related work

• Intrinsic MCP
• Zulip discussion associated with the MCP
• Accompanying PR implementing the intrinsic and the API

What happens now?

This issue contains an API change proposal (or ACP) and is part of the libs-api team feature lifecycle. Once this issue is filed, the libs-api team will review open proposals as capability becomes available. Current response times do not have a clear estimate, but may be up to several months.

Possible responses

The libs team may respond in various different ways. First, the team will consider the problem (this doesn't require any concrete solution or alternatives to have been proposed):

• We think this problem seems worth solving, and the standard library might be the right place to solve it.
• We think that this probably doesn't belong in the standard library.

Second, if there's a concrete solution:

• We think this specific solution looks roughly right, approved, you or someone else should implement this. (Further review will still happen on the subsequent implementation PR.)
• We're not sure this is the right solution, and the alternatives or other materials don't give us enough information to be sure about that. Here are some questions we have that aren't answered, or rough ideas about alternatives we'd want to see discussed.

Start with the proposed core::hint API and the driver source in lib.rs linked from the issue, then compare the accompanying PR implementing the intrinsic and wrapper API. Check how the API forwards to the intrinsic and how the supported-platform behavior is covered; done means the proposal is reviewed and an implementation is accepted.