rust-lang / rust-lang/rfcs

Add a str::from_utf8_prefix method

Open
#3,203 8 comments 8 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

T-libs
Dominant language
Markdown
Stars
6.6k
Forks
1.7k
Avg merge
16h 14m
Merged PRs (30d)
1

Description

I want to take a slice of bytes and get the longest prefix of the slice which is valid utf8. My use case is that I'm reading utf8 data from the network but the read buffer might not be full, so it might contain a partial codepoint at the end. Also, if the remote host sends invalid utf8 it would be useful to be able to process any valid utf8 they send beforehand.

It's trivial to implement this function using Utf8Error::valid_up_to, but it requires unsafe code to avoid performing validation twice:

fn from_utf8_prefix(bytes: &[u8]) -> &str {
    match str::from_uf8(bytes) {
        Ok(s) => s,
        Err(err) => {
            unsafe {
                str::from_utf8_unchecked(&bytes[..err.valid_up_to()])
            }
        },
    }
}

This is even what the example code for Utf8Error does.

Since the standard library already contains the machinery for attempting utf8 validation and getting the index at which it fails, and since forcing people to write unsafe code for something trivial is generally a bad thing, I think this function should exist in std::str along-side from_utf8 and friends.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the issue discussion and the linked Utf8Error::valid_up_to documentation, then review how this RFC repository handles proposals for std::str APIs. A complete contribution would need a resolved proposal for the from_utf8_prefix behavior and its interaction with invalid or incomplete UTF-8.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.