Better support for byte ordered reads and writes?
- Dominant language
- Rust
- Stars
- 4.1k
- Forks
- 339
- PR merge metrics
- No merged PRs in 30d
Description
Something that came up today was the question how to read and write bytes with using a certain endianness. @goto-bus-stop replied in chat with the following:
```rust
let mut bytes = [0; 4];
input.read_all(&mut bytes)?;
let num = u32::from_le_bytes(bytes);
```
However they also pointed out that using [`byteorder`](https://docs.rs/byteorder/1.3.2/byteorder/trait.ByteOrder.html) one could do:
```rust
let num = input.read_u32::()?;
```
Which seems quite nice. A port of this functionality exists for Tokio in the form of [`tokio-byteorder`](https://docs.rs/tokio-byteorder/0.1.0/tokio_byteorder/index.html). With support for the `futures::io::{AsyncRead, AsyncWrite}` [currently in the works](https://github.com/jonhoo/tokio-byteorder/pull/2).
## Design questions
People are currently already capable of reading and writing bytes with a certain endianness, without any issues. The hard parts are taken care of. However it doesn't quite feel _ergonomic_ yet. So what I'm wondering is if we could perhaps improve the status quo here somewhat by providing support for this out of the box.
Writing bytes is fortunately already a one-liner:
```rust
use async_std::prelude::*;
use async_std::io::{self, prelude::*};
#[async_std::main]
async fn main () -> io::Result<()> {
let mut stdout = io::stdout();
stdout.write_all(&12_u16.to_le_bytes()).await?;
Ok(())
}
```
### Byteorder inspired
But reading bytes isn't yet. We could probably do better here, and I see a few options. The first is to follow `byteorder`'s lead and add 16 methods on the Read trait, two `Endianness` enums, and a `NativeEndian` type alias:
```rust
use std::io::{self, Cursor, prelude::*, BigEndian};
#[async_std::main]
async fn main() -> io::Result<()> {
let mut reader = Cursor::new(vec![2, 5, 3, 0]);
assert_eq!(517, reader.read_u16::().await?);
assert_eq!(768, reader.read_u16::().await?);
}
```
### std inspired
Another option seems to be to add 48 new methods on the Read trait (3 endianness * 16 nums), and try to follow `std`'s naming conventions more closely:
```rust
use std::io::{self, Cursor, prelude::*, BigEndian};
#[async_std::main]
async fn main() -> io::Result<()> {
let mut reader = Cursor::new(vec![2, 5, 3, 0]);
assert_eq!(517, reader.read_u16_be().await?);
assert_eq!(768, reader.read_u16_be().await?);
}
```
### using traits
The third option, and I have no idea if this works (we should test this) is to add two new methods on the Read trait, and a trait that we implement for all number types so we can be generic over them, and the method knows how to decode them:
```rust
use std::io::{self, Cursor, prelude::*, BigEndian};
#[async_std::main]
async fn main() -> io::Result<()> {
let mut reader = Cursor::new(vec![2, 5, 3, 0]);
assert_eq!(517_u16, reader.read_be_bytes().await?);
assert_eq!(768_u16, reader.read_be_bytes().await?);
}
```
This last approach is somewhat iffy because it would show up in the function signature, which means we'd have to expose it (but wouldn't want people to implement it). Or we could make it a sealed trait, but I'm not a fan of doing that.
It seems like https://internals.rust-lang.org/t/pre-rfc-safe-transmute/11347 might be proposing a trait that could potentially cover this, but I'm unsure about the exact implications and relation to this. Maybe we should bring it up?
If we could find a way to make this work this would definitely be my preferred option, as it's easy to add a counterpart to `Write` as well (creating symmetry, and an even smaller one-liner). But that's a big *if* because there seem to be quite a few hurdles
## Conclusion
I've talked about the current state of reading and writing bytes from `async_std::io::{Read, Write}`, and explored possible directions to improve this.
This is not something we need to find a solution for immediately, but it's something that if we can figure out it'll make writing certain programs easier for sure. Thanks!
Contributor guide
Assessment
This issue has not been assessed yet.