[ptr] Model by-value transmutation separately from `Ptr`
- Dominant language
- Rust
- Stars
- 2.6k
- Forks
- 179
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 29
Description
*Authored by an AI agent acting on joshlf's behalf.*
## Overview
#3686 proposes treating `Ptr` as a capability over a memory region: it carries a referent type, an access discipline, alignment knowledge, and an admissible-state contract. We have also considered extending `Ptr` to represent values, potentially by adding a third `Aliasing` mode in which the `T` lives by value inside the `Ptr`.
I don't think values belong in `Aliasing`, or in `Ptr` at all.
Values and pointers share some transmutability facts, but they have different carrier semantics. A `Ptr` describes a *place*: moving the `Ptr` does not move its referent, and other capabilities to that place may survive a conversion. A Rust value is different: consuming the value consumes the source capability, moving the carrier may move the object, and the destination may be materialized in fresh storage with unrelated alignment.
I think we should keep `Ptr` place-based, keep by-value conversion as a separate operation, and share only the proof relations that are genuinely common to both.
## A value is not a third aliasing mode
It is tempting to model:
```rust
Aliasing = Shared | Exclusive | Value
```
and let `Ptr` store a `T` inline. The analogy with `Exclusive` is useful at first: both cases have unique access. But the analogy breaks at the points that matter for transmutation.
| Property | `Exclusive` `Ptr` | by-value `T` |
| --- | --- | --- |
| Referent survives moving the carrier | yes | not necessarily |
| A source/ancestor capability may become usable again | yes | no; the source is consumed |
| Destination uses the same address | yes | no requirement |
| Destination inherits source alignment constraints | yes, or alignment must be forgotten | no |
| Interior mutability creates aliasing obligations | yes | no |
| Shrinking may discard trailing bytes | not an exact reinterpretation | naturally |
| Source drop obligation survives as `Src` | yes for the place | no; it must be transferred or suppressed |
The second row is the most important.
Suppose an `Exclusive` `Ptr` is derived from an `&mut Src`. Consuming the `Ptr` does not permanently destroy the `Src` interpretation: the ancestor `&mut Src` may become usable again after the derived pointer dies. Therefore, writes through `Dst` must preserve whatever validity `Src` will later require.
This is why the exact-reinterpretation rule in #3686 includes a reverse preservation obligation such as:
```text
Q(Dst, DV) ⊆ Q(Src, SV)
```
when the destination can mutate the referent.
A by-value conversion has no corresponding obligation. Once `Src` has been consumed into `Dst`, there is no surviving capability that will later observe those bytes as a `Src`.
Encoding both cases in `Aliasing` would therefore make one parameter describe two independent questions:
1. how a persistent place may be accessed concurrently; and
2. whether the source interpretation survives the conversion at all.
We have already encountered a similar modeling problem in #1183. Earlier designs encoded `Box`, `Arc`, etc. as new aliasing modes, but that coupled source/container identity to shared-vs-exclusive access. The later `Source`-invariant design separated those concerns. Values are an even stronger reason not to put ownership or carrier kind in `Aliasing`.
## The common rule is about surviving capabilities
There is still a useful common model across references, values, and owning pointers:
> A reinterpretation must establish the destination interpretation and preserve every capability or obligation that survives the conversion.
Different carriers leave different things alive:
- For a borrowed `Ptr`, ancestor access may survive. Source-side writes, destination-side writes, and shared coexistence can therefore impose obligations in both directions.
- For a by-value `Src`, the source capability is consumed. No code can later access the object as `Src`, so pointer-specific ancestor-preservation obligations disappear.
- For `Box -> Box`, typed access as `Src` may disappear, but the allocation and eventual deallocation obligation survives. The conversion must therefore preserve the allocation properties required to reconstruct and drop the `Box`.
- For `Arc`, allocation ownership survives and other aliases may also survive, so more obligations remain.
This framing generalizes the model without forcing all carriers into one Rust type.
## By-value transmutation has a smaller proof
For a same-size by-value conversion:
```text
Src -> Dst
```
the relevant proof is roughly:
1. the source representation can be safely transferred;
2. the transferred state satisfies `Dst`'s validity requirements;
3. `Src`'s destructor is not run on the consumed representation; and
4. the resulting `Dst` owns the destination representation exactly once.
Unlike a `Ptr` reinterpretation, this does **not** inherently require:
- preserving `Src` validity after the conversion;
- compatibility between simultaneous `Src` and `Dst` views;
- `Immutable`;
- an alignment relationship between `Src` and `Dst`.
The last point is especially useful as a discriminator. A pointer reinterpretation operates on the same address, so destination alignment is constrained by the source place. A value conversion may materialize `Dst` in new destination storage, so `Dst` can receive whatever alignment it requires.
This matches the distinction that motivated #251: interior mutability constrains reference casts, but not equivalent by-value casts.
## The current value path already uses `Ptr` only as a temporary place
`try_transmute!` currently follows a shape like this:
```text
owned Src
|
| wrap in ManuallyDrop / ReadOnly
v
temporary place containing Src
|
| borrow that place
v
Ptr<'_, ...>
|
| validate destination
v
read out Dst, or reconstruct Src on failure
```
That decomposition makes conceptual sense. Validation needs a place from which it can inspect bytes, so the implementation temporarily creates a `Ptr`. But the input and output are still values; the `Ptr` is an implementation device over temporary storage, not the semantic carrier of the conversion.
I think we should preserve that distinction.
## Don't mirror the full `Ptr` validity lattice onto values
A separate type such as:
```rust
Value
```
where `V` is `Uninit`, `AsInitialized`, `Initialized`, or `Safe`, also looks attractive. I don't think the full lattice is sound or useful for freely movable values.
Some `Ptr` validity states describe the representation of a *fixed place*. In particular, `Initialized` and probably `AsInitialized` are not necessarily preserved by ordinary typed moves: initialized padding or initialized-but-invalid bytes need not survive a typed copy. #2354 already identifies this as the reason ordinary wrapper types cannot faithfully realize all of the current `Ptr` validity states.
That creates a basic problem for a movable abstraction:
```rust
let x: Value = ...;
let y = x;
```
An ordinary Rust move should preserve the type's invariant. For `Initialized`, we cannot generally promise that it does.
The other endpoints do not justify the parameter either:
- An ordinary `T` already represents ownership of a valid `T`; `Value` would mostly restate the Rust type system.
- `MaybeUninit` already represents owned uninitialized storage for sized values.
So I would not introduce a movable `Value` merely to make values look structurally similar to `Ptr`.
## A small internal `Value` may still be useful
There may still be an implementation case for a valid-only ownership guard:
```rust
struct Value {
// owns exactly one valid T and controls whether its destructor runs
}
```
Such a type could centralize the `ManuallyDrop` and success/error-path bookkeeping used by by-value conversions. Conceptually, it might provide operations like:
```rust
Value::new(t: T) -> Value
value.as_ptr()
-> Ptr<'_, ReadOnly, (Shared, Aligned, Safe)>
value.as_mut_ptr()
-> Ptr<'_, T, (Exclusive, Aligned, Safe)>
value.into_inner()
-> T
```
A successful conversion could consume/disarm the source `Value` and construct a `Dst`; a failed conversion could recover the original `Src`.
I would only add this abstraction if it materially simplifies implementation. It should not have `Aliasing`, `Alignment`, or `Validity` parameters. Its invariant would simply be that it owns one valid `T` and controls that value's drop obligation.
## What should be shared
The useful commonality is in the proof relations, not in the carrier type.
For example, the relation currently named `TransmuteFrom` describes a directional relationship between admissible source and destination states. That relationship can be useful whether those states occur in a persistent place or during a by-value conversion. This is another reason to view `TransmuteFrom` as something like a state-inclusion relation rather than as a pointer operation, as proposed in #3686.
Likewise, `TryFromBytes` validation can be shared. A value conversion may temporarily lend its storage to the existing `Ptr`-based validation machinery without making the value itself a `Ptr`.
Layout facts should also be shared where their contracts match. Carrier-specific obligations should remain carrier-specific.
## Proposed direction
I suggest that we make the following design choices explicit:
1. `Ptr` models a **place**. Its referent exists independently of the `Ptr` object and may have surviving ancestor or peer capabilities.
2. `Aliasing` describes access to that place. It should not encode by-value ownership, `Box` ownership, allocation source, or reconstruction rights.
3. By-value transmutation remains a separate carrier-specific operation (`transmute!`, `try_transmute!`, and their internal helpers).
4. We share state/transmutability and validation relations where their semantics are carrier-independent.
5. We do not introduce `Value` with the full `Ptr` validity lattice.
6. We consider an internal valid-only `Value` ownership guard only if it removes enough `ManuallyDrop`/drop-transfer/error-path complexity to justify itself.
This would refine the model from #3686 into two layers:
```text
common semantics:
typed interpretation + admissible state
reinterpretation preserves every surviving capability/obligation
carrier-specific semantics:
Ptr/reference -> place, lifetime, aliasing, alignment, ancestors
Box/etc. -> place plus allocation/source/deallocation obligations
value T -> ownership/drop transfer; no persistent aliasing relation
```
The goal is not to keep separate APIs for their own sake. It is to share the parts of the proof that are actually common without making `Ptr` carry distinctions that only exist because different carriers have different lifetime and ownership semantics.
Contributor guide
Research direction
Start with the transmute!, try_transmute!, Ptr, TransmuteFrom, and TryFromBytes entry points described in the issue, then compare their carrier and validation responsibilities. Done means an agreed design that keeps Ptr place-based, treats by-value transmutation separately, shares only carrier-independent proof relations, and avoids a full Value validity lattice.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design, security
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100