CrowdStrike / CrowdStrike/csproto
Allow overriding the message size limit of 2GB
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 140
- Forks
- 25
- Avg merge
- 13h 3m
- Merged PRs (30d)
- 3
Description
Title
Add a Decoder.SetMaxFieldLength() function to allow overriding the default limit of 2GB.
Description
The Decoder enforces a limit of 2GB on variable length fields like strings, bytes and nested messages.
We hit this limit for large snapshots in PowerDNS Lightning Stream as described in https://github.com/PowerDNS/lightningstream/issues/112
We propose adding a Decoder.SetMaxFieldLength() function to allow overriding this limit, while still retaining the default of 2GB when not overridden (PR incoming).
Additional Info
This limit is recommended by https://protobuf.dev/programming-guides/proto-limits/ as a way to guarantee interoperability.
It is also the only limit on that page that is not specified as being language dependent.
Go internally uses a 64-bit int for sizes on 64ibit platforms, and the actual length in the wire format is encoded in a varint that is decoded as a uint64 and can represent its full range, so there is no technical reason to impose a 2GB limit in a Go implementation.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the Decoder implementation and the existing 2GB limit for variable-length fields. Add the proposed SetMaxFieldLength entry point while preserving the default limit, then verify that strings, bytes, and nested messages honor an overridden value and retain the default when it is not set.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100