async-rs / async-rs/async-std

Some data is dropped on buffer boundary crossing when implementing minimal custom Stream of Lines over BufReader.

Open
#871 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
4.1k
Forks
339
PR merge metrics
No merged PRs in 30d

Description

I tried to implement minimalist `Stream` over lines in a `BufReader` using `read_line` method, but it drops some data when boundary of its buffer is crossed.
Using `async_std::io::Lines` does not reproduce the issue, as well as calling `read_line` in a loop does not reproduce it either.

Here is the code:
```rust
use std::{
pin::Pin,
task::{Context, Poll},
};

use async_std::{
fs::File,
io::{prelude::*, stdout, BufReader, Result},
prelude::*,
stream::Stream,
task,
};

fn main() -> Result<()> {
task::block_on(run())
}

async fn run() -> Result<()> {
let input = BufReader::with_capacity(284, File::open("pattern.txt").await?);
let mut lines = Lines::new(input);

while let Some(line) = lines.next().await {
stdout().write_all(line?.as_bytes()).await?;
}

Ok(())
}

pub struct Lines {
file: BufReader,
}

impl Lines {
pub fn new(file: BufReader) -> Self {
Self { file }
}
}

impl Stream for Lines {
type Item = Result;

fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {
let mut buf = String::new();
let mut line = self.file.read_line(&mut buf);
match Pin::new(&mut line).poll(cx) {
Poll::Pending => Poll::Pending,
Poll::Ready(Err(e)) => Poll::Ready(Some(Err(e.into()))),
Poll::Ready(Ok(0)) => Poll::Ready(None),
Poll::Ready(Ok(_)) => Poll::Ready(Some(Ok(buf))),
}
}
}
```

Here is the content of the file `patttern.txt`:
```
00 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
01 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
02 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
03 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
04 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
05 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
06 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
07 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
08 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
09 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
```

Here is the output of the sample program:
```
00 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
01 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
02 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
03 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
05 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
06 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
07 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
3456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
09 | 0123456789-ABCDEFGHIJKLMNOPQRSTUVWXYZ+abcdefghijklmnopqrstuvwxyz
```

Notes:
* Length of a single line is 70 bytes.
* Buffer size is set to 284 bytes (4 lines + 4 bytes).
* When buffer boundary is crossed (after first 4 lines), next 4 bytes are dropped (the remainder of the buffer) and following line contains only remaining 66 bytes.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.