apache / apache/beam

[Feature Request]: Support for efficient serialization of more built-in types

Open
#38,549 0 comments 0 reactions 0 assignees View on GitHub
new feature P2 python
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What would you like to happen?

Some very commonly used builtin types fallback to picklecoder right now, namely:
frozenset
datetime.datetime
datetime.date

It would be great to handle these specifically in FastPrimitivesCoder. I'm happy to contribute the code if the idea is OKed.

Also, if PEP 814 finds its way into python 3.15, it would be great to support frozendict as well. How does Beam generally handle something like that with a new python version?

### Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

### Issue Components

- [x] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Infrastructure
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Prism Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.