Fix Unicode string decoding in the dependency-free JSON parser
- Dominant language
- Rust
- Stars
- 53
- Forks
- 16
- Avg merge
- 4h 22m
- Merged PRs (30d)
- 46
Description
## Problem
`utils/src/json.rs` parses JSON strings byte by byte. Raw non-ASCII UTF-8 bytes are currently pushed as independent characters, and valid `\uXXXX` escapes are rejected.
This parser is used for Wave metadata such as the standard-library manifest, so valid JSON strings must round-trip without corruption.
## Scope
- Preserve raw UTF-8 text in JSON strings.
- Decode `\uXXXX` escapes.
- Combine valid UTF-16 surrogate pairs for code points outside the BMP.
- Reject malformed hexadecimal escapes and lone surrogates.
- Preserve the existing handling of quotes, slashes, control escapes, and unterminated strings.
## Tests
Add unit tests in the `utils` crate for:
- Raw Korean and emoji text.
- BMP Unicode escapes.
- A surrogate-pair escape.
- Invalid hex digits.
- Lone high and low surrogates.
- Parser/writer round trips.
## Completion criteria
- [ ] Valid Unicode JSON strings parse without corruption.
- [ ] Invalid Unicode escapes return an error rather than panicking.
- [ ] `cargo test --locked --all-targets --jobs 2` passes.
- [ ] No full serialization dependency is added.
Contributor guide
Research direction
Start in utils/src/json.rs and inspect the byte-by-byte JSON string parser, then review the existing utils crate tests. Add coverage for raw Korean and emoji text, BMP escapes, surrogate pairs, malformed escapes, lone surrogates, and parser/writer round trips. Done means valid Unicode round-trips, invalid escapes return errors without panicking, no serialization dependency is added, and cargo test --locked --all-targets --jobs 2 passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100