danielgtaylor / danielgtaylor/python-betterproto
Support for coded streams
- Dominant language
- Python
- Stars
- 1.8k
- Forks
- 234
- PR merge metrics
- No merged PRs in 30d
Description
C++ and Java implementations of protocol buffers have support for [writing and reading multiple messages](https://protobuf.dev/programming-guides/techniques/#streaming) via `CodedOutputStream` and `CodedInputStream` . This is also space-efficient because the size of each message can be represented as a `varint`.
I think having something similar to `CodedOutputStream` and `CodedInputStream` in `betterproto` would be useful. One use case would be when a Python tool uses `betterproto` to emit a binary stream of messages using this and then a C++/Java tool can use `CodedInputStream` to read the stream.
Related Java documentation:
* [CodedInputStream](https://protobuf.dev/reference/java/api-docs/com/google/protobuf/CodedInputStream.html)
* [CodedOutputStream](https://protobuf.dev/reference/java/api-docs/com/google/protobuf/CodedOutputStream)
Contributor guide
Research direction
Start by reviewing the linked protobuf streaming guidance and the Java CodedInputStream and CodedOutputStream documentation, then inspect how betterproto currently represents and serializes messages. Define an API for reading and writing length-delimited varint-framed messages, and verify that streams emitted by Python can be consumed by the documented C++ and Java implementations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100