Deserializing a flattened struct containing a `ByteBuf` fails to deserialize with invalid UTF-8.
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 5.6k
- Forks
- 670
- PR merge metrics
- No merged PRs in 30d
Description
As per the comment for deserialize_bytes, it is expected that you can deserialize a non-UTF-8 string into a ByteBuf without failure. However, with flattened structures, deserialize_map is called in place of deserialize_struct (see https://github.com/serde-rs/serde/issues/1529), meaning members are deserialized with deserialize_any bypassing deserialize_bytes. deserialize_any assumes values surrounded by quotation marks are valid UTF-8 strings and returns an error otherwise.
An example follows
use serde::{Deserialize, Serialize};
fn main() {
let a_success: A = serde_json::from_slice(b"{\"b\": {\"buf\": \"\xe5\x00\xe5\"}}").unwrap();
println!("A Success: {:?}", a_success);
let a_flat_success: AFlat = serde_json::from_slice(b"{\"buf\": \"abc\"}").unwrap();
println!("A Flat Success: {:?}", a_flat_success);
let a_flat_failure: AFlat = serde_json::from_slice(b"{\"buf\": \"\xe5\x00\xe5\"}").unwrap();
}
#[derive(Debug, Deserialize, Serialize)]
struct A {
b: B,
}
#[derive(Debug, Deserialize, Serialize)]
struct AFlat {
#[serde(flatten)]
b: B,
}
#[derive(Debug, Deserialize, Serialize)]
struct B {
buf: serde_bytes::ByteBuf,
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the example with A, AFlat, and serde_bytes::ByteBuf, then trace how flattened fields move through deserialize_map and deserialize_any instead of deserialize_struct and deserialize_bytes. Compare the behavior with the existing deserialize_bytes expectation; done means the flattened non-UTF-8 ByteBuf example deserializes successfully without regressing the non-flattened case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100