google / google/zerocopy

[ptr] Model by-value transmutation separately from `Ptr`

Open
#3,688 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2.6k
Forks
179
Avg merge
1d 19h
Merged PRs (30d)
29

Description

*Authored by an AI agent acting on joshlf's behalf.*

## Overview

#3686 proposes treating `Ptr` as a capability over a memory region: it carries a referent type, an access discipline, alignment knowledge, and an admissible-state contract. We have also considered extending `Ptr` to represent values, potentially by adding a third `Aliasing` mode in which the `T` lives by value inside the `Ptr`.

I don't think values belong in `Aliasing`, or in `Ptr` at all.

Values and pointers share some transmutability facts, but they have different carrier semantics. A `Ptr` describes a *place*: moving the `Ptr` does not move its referent, and other capabilities to that place may survive a conversion. A Rust value is different: consuming the value consumes the source capability, moving the carrier may move the object, and the destination may be materialized in fresh storage with unrelated alignment.

I think we should keep `Ptr` place-based, keep by-value conversion as a separate operation, and share only the proof relations that are genuinely common to both.

## A value is not a third aliasing mode

It is tempting to model:

```rust
Aliasing = Shared | Exclusive | Value
```

and let `Ptr` store a `T` inline. The analogy with `Exclusive` is useful at first: both cases have unique access. But the analogy breaks at the points that matter for transmutation.

| Property | `Exclusive` `Ptr` | by-value `T` |
| --- | --- | --- |
| Referent survives moving the carrier | yes | not necessarily |
| A source/ancestor capability may become usable again | yes | no; the source is consumed |
| Destination uses the same address | yes | no requirement |
| Destination inherits source alignment constraints | yes, or alignment must be forgotten | no |
| Interior mutability creates aliasing obligations | yes | no |
| Shrinking may discard trailing bytes | not an exact reinterpretation | naturally |
| Source drop obligation survives as `Src` | yes for the place | no; it must be transferred or suppressed |

The second row is the most important.

Suppose an `Exclusive` `Ptr` is derived from an `&mut Src`. Consuming the `Ptr` does not permanently destroy the `Src` interpretation: the ancestor `&mut Src` may become usable again after the derived pointer dies. Therefore, writes through `Dst` must preserve whatever validity `Src` will later require.

This is why the exact-reinterpretation rule in #3686 includes a reverse preservation obligation such as:

```text
Q(Dst, DV) ⊆ Q(Src, SV)
```

when the destination can mutate the referent.

A by-value conversion has no corresponding obligation. Once `Src` has been consumed into `Dst`, there is no surviving capability that will later observe those bytes as a `Src`.

Encoding both cases in `Aliasing` would therefore make one parameter describe two independent questions:

1. how a persistent place may be accessed concurrently; and
2. whether the source interpretation survives the conversion at all.

We have already encountered a similar modeling problem in #1183. Earlier designs encoded `Box`, `Arc`, etc. as new aliasing modes, but that coupled source/container identity to shared-vs-exclusive access. The later `Source`-invariant design separated those concerns. Values are an even stronger reason not to put ownership or carrier kind in `Aliasing`.

## The common rule is about surviving capabilities

There is still a useful common model across references, values, and owning pointers:

> A reinterpretation must establish the destination interpretation and preserve every capability or obligation that survives the conversion.

Different carriers leave different things alive:

- For a borrowed `Ptr`, ancestor access may survive. Source-side writes, destination-side writes, and shared coexistence can therefore impose obligations in both directions.
- For a by-value `Src`, the source capability is consumed. No code can later access the object as `Src`, so pointer-specific ancestor-preservation obligations disappear.
- For `Box -> Box`, typed access as `Src` may disappear, but the allocation and eventual deallocation obligation survives. The conversion must therefore preserve the allocation properties required to reconstruct and drop the `Box`.
- For `Arc`, allocation ownership survives and other aliases may also survive, so more obligations remain.

This framing generalizes the model without forcing all carriers into one Rust type.

## By-value transmutation has a smaller proof

For a same-size by-value conversion:

```text
Src -> Dst
```

the relevant proof is roughly:

1. the source representation can be safely transferred;
2. the transferred state satisfies `Dst`'s validity requirements;
3. `Src`'s destructor is not run on the consumed representation; and
4. the resulting `Dst` owns the destination representation exactly once.

Unlike a `Ptr` reinterpretation, this does **not** inherently require:

- preserving `Src` validity after the conversion;
- compatibility between simultaneous `Src` and `Dst` views;
- `Immutable`;
- an alignment relationship between `Src` and `Dst`.

The last point is especially useful as a discriminator. A pointer reinterpretation operates on the same address, so destination alignment is constrained by the source place. A value conversion may materialize `Dst` in new destination storage, so `Dst` can receive whatever alignment it requires.

This matches the distinction that motivated #251: interior mutability constrains reference casts, but not equivalent by-value casts.

## The current value path already uses `Ptr` only as a temporary place

`try_transmute!` currently follows a shape like this:

```text
owned Src
|
| wrap in ManuallyDrop / ReadOnly
v
temporary place containing Src
|
| borrow that place
v
Ptr<'_, ...>
|
| validate destination
v
read out Dst, or reconstruct Src on failure
```

That decomposition makes conceptual sense. Validation needs a place from which it can inspect bytes, so the implementation temporarily creates a `Ptr`. But the input and output are still values; the `Ptr` is an implementation device over temporary storage, not the semantic carrier of the conversion.

I think we should preserve that distinction.

## Don't mirror the full `Ptr` validity lattice onto values

A separate type such as:

```rust
Value
```

where `V` is `Uninit`, `AsInitialized`, `Initialized`, or `Safe`, also looks attractive. I don't think the full lattice is sound or useful for freely movable values.

Some `Ptr` validity states describe the representation of a *fixed place*. In particular, `Initialized` and probably `AsInitialized` are not necessarily preserved by ordinary typed moves: initialized padding or initialized-but-invalid bytes need not survive a typed copy. #2354 already identifies this as the reason ordinary wrapper types cannot faithfully realize all of the current `Ptr` validity states.

That creates a basic problem for a movable abstraction:

```rust
let x: Value = ...;
let y = x;
```

An ordinary Rust move should preserve the type's invariant. For `Initialized`, we cannot generally promise that it does.

The other endpoints do not justify the parameter either:

- An ordinary `T` already represents ownership of a valid `T`; `Value` would mostly restate the Rust type system.
- `MaybeUninit` already represents owned uninitialized storage for sized values.

So I would not introduce a movable `Value` merely to make values look structurally similar to `Ptr`.

## A small internal `Value` may still be useful

There may still be an implementation case for a valid-only ownership guard:

```rust
struct Value {
// owns exactly one valid T and controls whether its destructor runs
}
```

Such a type could centralize the `ManuallyDrop` and success/error-path bookkeeping used by by-value conversions. Conceptually, it might provide operations like:

```rust
Value::new(t: T) -> Value

value.as_ptr()
-> Ptr<'_, ReadOnly, (Shared, Aligned, Safe)>

value.as_mut_ptr()
-> Ptr<'_, T, (Exclusive, Aligned, Safe)>

value.into_inner()
-> T
```

A successful conversion could consume/disarm the source `Value` and construct a `Dst`; a failed conversion could recover the original `Src`.

I would only add this abstraction if it materially simplifies implementation. It should not have `Aliasing`, `Alignment`, or `Validity` parameters. Its invariant would simply be that it owns one valid `T` and controls that value's drop obligation.

## What should be shared

The useful commonality is in the proof relations, not in the carrier type.

For example, the relation currently named `TransmuteFrom` describes a directional relationship between admissible source and destination states. That relationship can be useful whether those states occur in a persistent place or during a by-value conversion. This is another reason to view `TransmuteFrom` as something like a state-inclusion relation rather than as a pointer operation, as proposed in #3686.

Likewise, `TryFromBytes` validation can be shared. A value conversion may temporarily lend its storage to the existing `Ptr`-based validation machinery without making the value itself a `Ptr`.

Layout facts should also be shared where their contracts match. Carrier-specific obligations should remain carrier-specific.

## Proposed direction

I suggest that we make the following design choices explicit:

1. `Ptr` models a **place**. Its referent exists independently of the `Ptr` object and may have surviving ancestor or peer capabilities.
2. `Aliasing` describes access to that place. It should not encode by-value ownership, `Box` ownership, allocation source, or reconstruction rights.
3. By-value transmutation remains a separate carrier-specific operation (`transmute!`, `try_transmute!`, and their internal helpers).
4. We share state/transmutability and validation relations where their semantics are carrier-independent.
5. We do not introduce `Value` with the full `Ptr` validity lattice.
6. We consider an internal valid-only `Value` ownership guard only if it removes enough `ManuallyDrop`/drop-transfer/error-path complexity to justify itself.

This would refine the model from #3686 into two layers:

```text
common semantics:
typed interpretation + admissible state
reinterpretation preserves every surviving capability/obligation

carrier-specific semantics:
Ptr/reference -> place, lifetime, aliasing, alignment, ancestors
Box/etc. -> place plus allocation/source/deallocation obligations
value T -> ownership/drop transfer; no persistent aliasing relation
```

The goal is not to keep separate APIs for their own sake. It is to share the parts of the proof that are actually common without making `Ptr` carry distinctions that only exist because different carriers have different lifetime and ownership semantics.

Contributor guide

Open the contributing guide

Research direction

Start with the transmute!, try_transmute!, Ptr, TransmuteFrom, and TryFromBytes entry points described in the issue, then compare their carrier and validation responsibilities. Done means an agreed design that keeps Ptr place-based, treats by-value transmutation separately, shares only carrier-independent proof relations, and avoids a full Value validity lattice.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, security
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.