danielgtaylor / danielgtaylor/python-betterproto

Support for coded streams

Open
#482 0 comments 1 reaction 0 assignees View on GitHub
enhancement medium
Dominant language
Python
Stars
1.8k
Forks
234
PR merge metrics
No merged PRs in 30d

Description

C++ and Java implementations of protocol buffers have support for [writing and reading multiple messages](https://protobuf.dev/programming-guides/techniques/#streaming) via `CodedOutputStream` and `CodedInputStream` . This is also space-efficient because the size of each message can be represented as a `varint`.

I think having something similar to `CodedOutputStream` and `CodedInputStream` in `betterproto` would be useful. One use case would be when a Python tool uses `betterproto` to emit a binary stream of messages using this and then a C++/Java tool can use `CodedInputStream` to read the stream.

Related Java documentation:
* [CodedInputStream](https://protobuf.dev/reference/java/api-docs/com/google/protobuf/CodedInputStream.html)
* [CodedOutputStream](https://protobuf.dev/reference/java/api-docs/com/google/protobuf/CodedOutputStream)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the linked protobuf streaming guidance and the Java CodedInputStream and CodedOutputStream documentation, then inspect how betterproto currently represents and serializes messages. Define an API for reading and writing length-delimited varint-framed messages, and verify that streams emitted by Python can be consumed by the documented C++ and Java implementations.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.