apache / apache/beam

[Feature Request]: Serialize Python user types with Beam schemas

Open
#22,082 3 comments 1 reaction 0 assignees View on GitHub
core new feature P3 python schemas
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What would you like to happen?

#16558 proposed adding an `sdk_options` attribute to the Beam schema proto allowing SDKs to encode serialized user types. We should revive that PR, and use it to serialize the (Python) user type that will be used to represent the schema (or from which the schema was inferred).

This will enable users to utilize methods on their types, and will make it simpler for us to add support for inferring from other user types (dataclasses, TypedDicts, etc..).

### Issue Priority

Priority: 2

### Issue Component

Component: sdk-py-core

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.