Tracking issue: Serde and `#[derive(AvroSchema)]` improvements
- Dominant language
- Rust
- Stars
- 130
- Forks
- 62
- Avg merge
- 11h
- Merged PRs (30d)
- 45
Description
Our current Serde and derive implementation works, but has room for improvement. This is a unordered list of improvements I'm planning to work on.
- [x] Add documentation for the derive macro (https://github.com/apache/avro-rs/tree/feat/improve_documentation)
- [x] All supported attributes
- [x] Compatibility and interaction with Serde
- [ ] Make `avro(default)` (no specific value) use `Default::::default()` instead of always using `None`
- [ ] Add `append_ser` support to `GenericSingleObjectWriter`/`to_avro_datum`
- [x] Move all Serde related documentation and functions to the `serde` module (and make that public), so the main documentation is less cluttered (https://github.com/apache/avro-rs/tree/feat/improve_documentation)
- [x] Also add a short explainer of how Avro works to the Serde part
- [ ] Schema aware deserialisation (see #237)
- [ ] Implement `SerializeMap::serialize_entry`
- [ ] Allow disabling getting the rustdoc via `doc = false`
- [ ] In the derived `get_schema_with_ctxt` validate that the schema is valid (https://github.com/apache/avro-rs/issues/365#issuecomment-3730627801)
- [x] Support newtype structs
- [x] Support tuples
- [x] Make clear that errors come from AvroSchema, as that is not visible to the user
- [x] Support `char` (as a `Schema::String` or a `Schema::Fixed(4)`)
- [ ] Cleanup derive tests to be more like `avro_3687_basic_enum_with_default` instead of `test_avro_3709_record_field_attributes`
- [ ] Implement `AvroSchemaComponent` for all types that `Serde` implements [`Serialize`](https://docs.rs/serde/latest/serde/trait.Serialize.html) for
- [x] Allow overwriting the schema type for a field (i.e. `avro(type = "bytes")`, see #350) (#397)
- [x] Support `#[serde(transparent)]` (#398)
- [x] Make `AvroSchemaComponent` always available (#394)
- [x] Try to use as many Serde attributes as possible so that user doesn't have duplicate attributes (`avro` + `serde`) (#377)
- [x] Check Serde attributes so we can error on incompatible attributes (i.e. `skip_serializing_if` without a `avro(default)` (#377)
Future work:
- [ ] Support simple closures for the `with` attribute (i.e. `#[avro(with = || Schema::Bytes)]`)
Not possible:
- [ ] Use LazyLock to only create the schema once (inspired by https://github.com/lerouxrgd/rsgen-avro/pull/90)
- This does not work for types with generics or lifetimes, as the `const`/`static` would be shared between all implementations
- It also doesn't work for types without generics or lifetimes as the output can change based on the `names` and `enclosing_namespace` arguments
- [ ] Add a test for every derive that runs `T::get_schema()`
- This does not work for types with generics or lifetimes, we don't know what bounds are required on the types so we can't name the type.
- This also doesn't work for types without generics of lifetimes, because derives are only allowed to emit `impl` blocks, so we can't emit a test function
Contributor guide
No contributing guide indexed for this repository
Research direction
This is a broad tracking issue with several unchecked Serde and derive improvements rather than one scoped task. Start by choosing a specific item, then read the public serde module, get_schema_with_ctxt, and the mentioned derive tests avro_3687_basic_enum_with_default and test_avro_3709_record_field_attributes. Done depends on the selected checklist item and its corresponding tests passing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100