seanmonstar / seanmonstar/httparse

Parse from noncontiguous byte slices?

Open
#74 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
713
Forks
135
PR merge metrics
No merged PRs in 30d

Description

I am building a HTTP server on top of tokio that needs to perform minimal memory allocations per client TCP connection, regardless of how many HTTP requests are received through the connection. Therefore I have chosen to use a fixed-size circular buffer to store the raw request data read from the wire, and now I am trying to use httparse to parse the request information. The problem I have run into is that the Request.parse function takes in a single &[u8], but because I'm using a circular buffer I have two slices - one for the bytes in the remainder of the buffer, and one (optionally) for the bytes which wrapped around to the front of the buffer. This two-buffer approach works very well with vectored IO reads, but not so well so far with httparse.

At first I was hoping I that httparse's Request type would be persistent, so I could call parse in turn for both of the slices. But that appears to not be how the API works - it expects the one slice to have all the data, and when you call it a second time the same data should still be present, only with more added to the end.

Consequently, the only way I can find to use httparse today is to perform a copy of the data from the circular buffer into a secondary contiguous buffer. But the cost of such copying is potentially significant and I'd prefer to avoid it where possible. How feasible would it be to add some sort of parse_vectored function to httpparse which takes a &[std::io::IoSlice]?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Request.parse and the existing push-parser behavior for a single &[u8]. Review how the circular buffer exposes its two slices, then define the proposed parse_vectored API around std::io::IoSlice. Done means split request data can be parsed incrementally without copying while preserving the existing semantics.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.