wavefnd / wavefnd/Wave

Fix Unicode string decoding in the dependency-free JSON parser

Open
#351 0 comments 0 reactions 0 assignees View on GitHub
bug good first issue
Dominant language
Rust
Stars
53
Forks
16
Avg merge
4h 22m
Merged PRs (30d)
46

Description

## Problem

`utils/src/json.rs` parses JSON strings byte by byte. Raw non-ASCII UTF-8 bytes are currently pushed as independent characters, and valid `\uXXXX` escapes are rejected.

This parser is used for Wave metadata such as the standard-library manifest, so valid JSON strings must round-trip without corruption.

## Scope

- Preserve raw UTF-8 text in JSON strings.
- Decode `\uXXXX` escapes.
- Combine valid UTF-16 surrogate pairs for code points outside the BMP.
- Reject malformed hexadecimal escapes and lone surrogates.
- Preserve the existing handling of quotes, slashes, control escapes, and unterminated strings.

## Tests

Add unit tests in the `utils` crate for:

- Raw Korean and emoji text.
- BMP Unicode escapes.
- A surrogate-pair escape.
- Invalid hex digits.
- Lone high and low surrogates.
- Parser/writer round trips.

## Completion criteria

- [ ] Valid Unicode JSON strings parse without corruption.
- [ ] Invalid Unicode escapes return an error rather than panicking.
- [ ] `cargo test --locked --all-targets --jobs 2` passes.
- [ ] No full serialization dependency is added.

Contributor guide

Open the contributing guide

Research direction

Start in utils/src/json.rs and inspect the byte-by-byte JSON string parser, then review the existing utils crate tests. Add coverage for raw Korean and emoji text, BMP escapes, surrogate pairs, malformed escapes, lone surrogates, and parser/writer round trips. Done means valid Unicode round-trips, invalid escapes return errors without panicking, no serialization dependency is added, and cargo test --locked --all-targets --jobs 2 passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.