google / google/zerocopy

Provide the ability to zero padding bytes and return `&Initialized<T>`

Open
#494 5 comments 1 reaction 0 assignees View on GitHub
compatibility-nonbreaking
Dominant language
Rust
Stars
2.6k
Forks
179
Avg merge
1d 19h
Merged PRs (30d)
29

Description

*Note: We likely won't do this. See the conversation below for the full context, but TL;DR: We'd need for any `KnownLayout` type to require that all of its fields also implement `KnownLayout`, which we don't currently require. We expect that the `freeze` language feature will land soon enough that it's not worth worsening `KnownLayout`'s UX to support this use case.*

## Progress

- [ ] Update this issue description per [this comment](https://github.com/google/zerocopy/issues/494#issuecomment-2140902501)
- [ ] Update `KnownLayout` to require that fields implement `KnownLayout` too
- [ ] Do one of the following:
- [ ] Decide that the `freeze` intrinsic ([RFC 3605](https://github.com/rust-lang/rfcs/pull/3605)) will land and stabilize soon enough that we can rely on it instead; relax `KnownLayout` to not be recursive (and watch out for [#1162](https://github.com/google/zerocopy/issues/1162))
- [ ] Use the recursive `KnownLayout` requirement to implement this design

## Details

Issues like [this one](https://bugs.fuchsia.dev/p/fuchsia/issues/detail?id=135123) demonstrate that it is sometimes useful to access the bytes of a type which cannot implement `AsBytes`. In these cases, it should be sound to:
- Recursively zero any inter-field padding bytes
- Provide access to the bytes of the object as an `&Initialized` where `Initialized: IntoBytes` even when `T: !IntoBytes`

We would need to teach `KnownLayout` to be able to zero padding, e.g.:

```rust
pub unsafe trait KnownLayout {
fn zero_padding(&mut self) -> &mut Initialized;
}

#[repr(transparent)]
pub struct Initialized {
// INVARIANT: Every byte in `inner` is initialized. Note that this implies
// that an `Initialized` cannot be moved by value unless `T: IntoBytes`
// since typed copies de-initialize padding bytes.
inner: T,
}

unsafe impl IntoBytes for Initialized {}

impl Deref for Initialized { ... }

// INVARIANT: Since `T: IntoBytes`, any value that is written via this impl
// has no padding bytes, and so will not invalidate the invariant that all of
// `inner`'s bytes are initialized.
impl DerefMut for Initialized { ... }

// TODO: Provide field projection
```

The only requirement for a type supporting this operation is that we know where its padding bytes are. The public API for this type could be in `KnownLayout`.

As of this writing, `KnownLayout` does not require that a type's fields also be `KnownLayout`. We are planning to add that requirement in order to support this design.

### Open questions

- What if we want to copy from a `&T` (which we can't modify) into a buffer while initializing any padding bytes in the destination [like musli-zerocopy does](https://github.com/udoprog/musli/blob/391770fe373d8b36d0a498f78458613f7b6dd19e/crates/musli-zerocopy/src/buf/mod.rs#L256-L267)? See [this discussion](https://github.com/google/zerocopy/issues/523#issuecomment-1771121721).

### Related & prior art

- https://github.com/rust-lang/rfcs/pull/3605

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.