otto-de / otto-de/api-guidelines
[define] usage of CloudEvents' `dataschema` conflicts with the Confluent wire format
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 50
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Description
(i) This issue has been manually transferred from a former internal repository, as a private repository issue cannot be transferred to a public repository.
Context
CloudEvents' dataschema should contain a reference to the schema that has been used to write the cloud event. If the Confluent schema registry and it's SerDes is used, this information is encoded using the Confluent wire format directly in the key/value of the record.
Currently, it is not possible to write the schema ID into a custom record header when using the Confluent Schema Registry SerDes. An open issue exists: https://github.com/confluentinc/schema-registry/issues/881
Other SerDe Libraries support writing to a custom header: https://www.apicur.io/registry/docs/apicurio-registry/2.2.x/getting-started/assembly-configuring-kafka-client-serdes.html#registry-serdes-config-props_registry
Therefore we need to define how this should be handled.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the CloudEvents dataschema specification and the linked Confluent wire-format documentation, then compare them with the referenced Confluent Schema Registry and Apicurio SerDes behavior. The work is done when the guidelines clearly define how dataschema should be represented when schema information is embedded in record keys or values rather than written to a custom header.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kafka
- Domain
- api, distributed-systems
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100