apache / apache/avro-rs

(De)serializer from/to JSON

Open
#518 5 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
130
Forks
62
Avg merge
11h
Merged PRs (30d)
45

Description

Original issue #449 by @PookieBuns

Currently we only support (de)serializing from/to JSON via the `TryFrom for avro::types::Value` and `TryFrom for serde_json::Value` implementations.
These implementations are not according to the spec:
```rust
#[test]
fn avro_rs_518_union_int_is_json_int_value() {
let v = Value::Union(1, Box::new(Value::Int(42)));
let json: JsonValue = v.try_into().unwrap();
assert_eq!(json, json!({"int": 42}));
}
#[test]
fn avro_rs_518_union_enum_is_json_enum_name_value() {
let v = Value::Union(1, Box::new(Value::Enum(5, "Problematic")));
let json: JsonValue = v.try_into().unwrap();
// The Value::Enum doesn't contain the enum name (some_name) so can't do the correct thing
assert_eq!(json, json!({"some_name": "Problematic"}));
}
```
Both these tests will fail. The first test can be fixed using the `TryFrom` implementations, but the second test cannot be fixed as `avro::types::Value` does not contain the necessary information for named types.

If we decide to support the JSON encoding of Avro, I suggest the following:
1. Create a new module `json` that will contain the `from_reader(reader, schema)`/`to_writer(writer, schema, value)`.
2. Create a new serializer and deserializer that wrap around `serde_json`'s serializer and deserializer, using the schema to call the right functions on `serde_json`'s serializer and deserializer.
- The alternative is (de)serializing to `serde_json::Value` first, and then implementing our serializer and deserializer around that. However, this is really slow.
- If there is a usecase for (de)serializing to `serde_json::Value`, support can always be added later with a `to_value/from_value` function (it can reuse the logic of `SchemaAwareRecordFieldDefault` for the serialisation part).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the existing TryFrom and TryFrom implementations and the two avro_rs_518 tests in the issue. Then assess the proposed json module with schema-aware from_reader/to_writer APIs and serializers. Done should include Avro-spec-compatible JSON encoding, including unions and named types, with tests covering the shown cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.