(De)serializer from/to JSON
- Dominant language
- Rust
- Stars
- 130
- Forks
- 62
- Avg merge
- 11h
- Merged PRs (30d)
- 45
Description
Original issue #449 by @PookieBuns
Currently we only support (de)serializing from/to JSON via the `TryFrom for avro::types::Value` and `TryFrom for serde_json::Value` implementations.
These implementations are not according to the spec:
```rust
#[test]
fn avro_rs_518_union_int_is_json_int_value() {
let v = Value::Union(1, Box::new(Value::Int(42)));
let json: JsonValue = v.try_into().unwrap();
assert_eq!(json, json!({"int": 42}));
}
#[test]
fn avro_rs_518_union_enum_is_json_enum_name_value() {
let v = Value::Union(1, Box::new(Value::Enum(5, "Problematic")));
let json: JsonValue = v.try_into().unwrap();
// The Value::Enum doesn't contain the enum name (some_name) so can't do the correct thing
assert_eq!(json, json!({"some_name": "Problematic"}));
}
```
Both these tests will fail. The first test can be fixed using the `TryFrom` implementations, but the second test cannot be fixed as `avro::types::Value` does not contain the necessary information for named types.
If we decide to support the JSON encoding of Avro, I suggest the following:
1. Create a new module `json` that will contain the `from_reader(reader, schema)`/`to_writer(writer, schema, value)`.
2. Create a new serializer and deserializer that wrap around `serde_json`'s serializer and deserializer, using the schema to call the right functions on `serde_json`'s serializer and deserializer.
- The alternative is (de)serializing to `serde_json::Value` first, and then implementing our serializer and deserializer around that. However, this is really slow.
- If there is a usecase for (de)serializing to `serde_json::Value`, support can always be added later with a `to_value/from_value` function (it can reuse the logic of `SchemaAwareRecordFieldDefault` for the serialisation part).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the existing TryFrom and TryFrom implementations and the two avro_rs_518 tests in the issue. Then assess the proposed json module with schema-aware from_reader/to_writer APIs and serializers. Done should include Avro-spec-compatible JSON encoding, including unions and named types, with tests covering the shown cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100