RustPython / RustPython/RustPython

csv: support non-ASCII Unicode dialect characters

Open
#8,408 1 comment 0 reactions 1 assignee View on GitHub

@doma17 is already working on this.

Since Aug 24, 2026.

Dominant language
Rust
Stars
22.4k
Forks
1.5k
Avg merge
16h 33m
Merged PRs (30d)
165

Description

Summary

Add support for single non-ASCII Unicode code points in CSV dialect character attributes, matching CPython behavior.

Rationale

csv dialect options such as delimiter, quotechar, and escapechar accept exactly one Unicode character in CPython. RustPython currently stores these attributes as u8 in crates/stdlib/src/csv.rs, so valid one-character values outside the byte range (for example, ) are rejected.

Affected area

  • crates/stdlib/src/csv.rs
  • CSV dialect parsing, validation, reader, and writer configuration

Required changes

Replace the byte-only representation and parsing path for dialect characters with a representation that preserves a full Unicode code point, then propagate that support through the CSV reader and writer implementation. Keep existing validation that rejects empty and multi-character values, while reporting conversion/representation failures accurately.

Acceptance criteria

  • delimiter, quotechar, and escapechar accept a single non-ASCII Unicode code point such as .
  • Empty and multi-character strings remain rejected according to CPython-compatible behavior.
  • Reader and writer behavior correctly handles the supported Unicode dialect characters.
  • Relevant RustPython CSV tests cover successful non-ASCII dialect-character usage and invalid lengths.

Backlinks

Requested by @hyoinandout.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.