linkedin / linkedin/goavro

How does goavro deal with complex nested types?

Open
#102 13 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Go
Stars
1.1k
Forks
232
PR merge metrics
No merged PRs in 30d

Description

Have a look at [this StackOverflow answer,](https://stackoverflow.com/a/29491401/32914) which depicts a single Avro schema as an array in which multiple record types are declared, which can reference each other.

This is an ideal format for describing complex structs that contain other structs, but it looks like `goavro` doesn't know how to support this. It expects any array to be a union type, so trying to register a codec for this wouldn't work. The obvious workaround would be to register them as distinct types, but doesn't appear that there's any way to do that either. While the codec building code uses a map internally to keep track of codecs for sub-types, it's created anew for each new codec and there doesn't appear to be a mechanism to say "create a new codec for this schema and here's a list of existing codecs for the struct types it references."

So what's the correct way to deal with structs that contain other structs (or slices of other structs)?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the codec-building code and the schema handling described in the issue, then compare it with the linked StackOverflow schema containing multiple named record types. Determine how referenced structs and slices of structs should be registered and resolved. Done means goavro can encode and decode complex nested records without treating the schema array as a union.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.