ga4gh / ga4gh/phenopacket-schema
Update to the latest Protobuf version
- Dominant language
- Java
- Stars
- 101
- Forks
- 34
- PR merge metrics
- No merged PRs in 30d
Description
## Task
Discuss and decide on updating to the latest Protobuf version.
## Background
- Moving to the latest versions of Protocol Buffers will re-enable the use of the optional label to explicitly mark fields, addressing several open ticket requests.
- Investigation and testing are needed to determine whether adopting newer Protobuf features would provide meaningful benefits and whether any resulting changes would affect versioning or backward compatibility.
- Our repository is currently using Protobuf 'proto3' version 3.20.3. Compared to 'proto2', 'proto3' features the following:
- Dropped `required` (only `optional` and `repeated`).
- Every field has an implicit default (`0` for ints, `false` for booleans, empty string for text)).
- Simpler syntax: no defaults in `.proto` files (they’re implied).
- proto 2's Extensions are mostly replaced with `Any.`
- Better JSON support, better interoperability.
- Diasmbiguate: the `phenopacket-tools` repository, claims in its `pom.xml` that we're using version, 3.21.7)
- As of May 2026, the latest stable release of the Protocol Buffers compiler (`protoc`) and core libraries is v35.0.
- Because of how Google versions its language-specific packages across different package managers, you will see this release labeled slightly differently depending on the ecosystem you are targeting:
- Java / C++ (Maven/RubyGems): 4.35.0
- Python (PyPI): 7.35.0
- Protoc / GitHub Releases: v35.0
- Once we make a decision to update, we must also update the Phenopackets reference implementation.
## Thoughts on updating to Protobuf Editions
Historically, Protobuf required us to choose between two rigid syntax versions: `proto2` and `proto3`. Each version had hardcoded behaviors—for example, how field presence was tracked, how default values were handled, or whether enums were open or closed. If you wanted a specific behavior from `proto2`, but your file was declared as proto3, you were generally out of luck. Protobuf Editions completely overhauls this model. They eliminate the rigid `proto2` vs. `proto3` binary and replace it with a Features-based system. An "Edition" is essentially just a packaged collection of default settings for these features. https://protobuf.dev/editions/overview/
**1. Declaration Change:** At the top of the`.proto` file, you no longer declare a syntax. Instead, you declare the year of the edition you are targeting.
Old way:
`syntax = "proto3";`
New Way
`edition = "2023";` // or "2024", etc.
**2. Features Define Behavior:** Instead of the compiler deciding how your fields work based on the word "proto3", behavior is now controlled by explicit feature flags.
- For example, in `proto3`, the default behavior for tracking whether a field was actually set by a user (field presence) was "implicit." In Editions, you can control this using `features.field_presence`.
- If an Edition changes a default behavior, but you want to keep the old behavior, you simply override the feature setting. This allows you to mix and match `proto2` and `proto3` semantics within the exact same file.
**3. Lexical Scoping (Granular Control):** Under the old system, `syntax = "proto3"` applied to the entire file. Under the Editions model, you can define behaviors globally but override them at the individual field level using options.
`edition = "2024";`
`// Set a file-wide default: all enums in this file are CLOSED (a proto2 behavior)`
`option features.enum_type = CLOSED;`
`message Player {`
`// Override field presence for just this specific field`
`int32 id = 1 [features.field_presence = IMPLICIT];`
`enum Status {`
`// Override the file-wide default for just this enum`
`option features.enum_type = OPEN;`
`STATUS_UNSPECIFIED = 0;`
`STATUS_ACTIVE = 1;`
`}`
`}`
**4. Incremental evolution:**
The biggest problem with the jump from `proto2` to `proto3` was that it fractured the ecosystem. Moving a massive codebase from one to the other was very disruptive. Google designed Editions (heavily inspired by Rust editions) to fix this. New editions are planned to be released roughly once a year. When a new edition is released, it does not introduce entirely new binary behaviors; it simply changes the defaults of existing features. This means you can incrementally ratchet your codebase forward, updating one `.proto` file to a new edition at a time without breaking compatibility with other files using older editions.
Sources:
https://github.com/protocolbuffers/protobuf/blob/main/docs/design/editions/what-are-protobuf-editions.md
https://protobuf.dev/editions/overview/
Contributor guide
Research direction
Start by inspecting the repository's .proto files and the phenopacket-tools pom.xml reference mentioned in the issue, then check the current protoc and Protobuf library versions. Determine whether an upgrade or Editions migration is compatible with existing schemas and generated clients. Done means documenting the decision, compatibility impact, and any required reference-implementation updates.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100