NethermindEth / NethermindEth/pluto
Extract shared Protobuf conversion helpers; standardize prost `Option`/default handling
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 8
- Forks
- 5
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 37
Description
Summary
proto3 presence semantics as prost surfaces them (every message field is Option, scalars silently default) are handled ad hoc per crate:
Duty ↔ pbcore::Dutyis implemented three times with three failure semantics: fallibleTryFromincore/src/types.rs#L282-L304; a lossy infallibleduty_from_protoinconsensus qbft/msg.rs#L370-L382(None→Duty::new(0, Unknown)); and another infallible pair inpriority/src/prioritiser.rs#L73-L88. Two share the nameduty_from_protowith different tolerance.- ~11
.ok_or(...)unwrap sites and 15+Some(...)wrapping sites for message fields; duplicatedBTreeMap<String,_> ↔ HashMap<PubKey,_>transcode loops and empty-map-means-invalid checks betweencore/types.rsandcore/unsigneddata.rs. - Zero-value conventions are hand-rolled per site: "absent vs zero hash" (
msg.rs#L330-L337), "empty bytes means missing signature" (#L315-L317). cluster/build.rsinlines its ownprost_build::Configinstead of usingpluto-build-proto, silently skippingenable_type_names()and the lint-header injection every other crate gets.
Message-field presence is proto3, not prost — any generator carries the same absent-vs-default ambiguity, so the leverage is in centralizing the handling, not switching generators.
One default-value problem is prost-specific and needs a different fix. prost omits a map entry's key or value field when it equals the type default, where protoc and Go always emit both. Reference bytes from protoc 32.1, byte-identical to Go:
| value | protoc / Go |
prost |
|---|---|---|
UnsignedDataSet{"0xaa": b""} |
0a080a04307861611200 |
0a060a0430786161 |
UnsignedDataSet{"": b""} |
0a040a001200 |
0a00 |
map<string,int32>{"a": 0} |
0a050a01611000 |
0a030a0161 |
hash_proto SSZ-hashes those bytes into the QBFT consensus value hash, so a mixed pluto/charon cluster derives different hashes for an identical value and fails to agree with nothing erroring. It is settled upstream — prost#99 closed as working-as-intended, the omission being spec-permitted even though no Google implementation does it — so the correction has to live on pluto's side. ParSignedDataSet has the same exposure, latent only because it never reaches hash_proto.
Proposed change
- One conversion module in
core(relates #127): a single fallibleDutyconversion (callers that want lossy behavior make it explicit), afn required<T>(field: Option<T>, name: &'static str) -> Result<T, ProtoFieldError>helper replacing theok_orboilerplate, and the shared map-transcode helpers. - Document the zero-value conventions (zero hash = absent, empty bytes = absent) once, next to the helpers.
- Correct map-entry encoding once for every message rather than per hashing call site, and guard it with a descriptor-driven test: walk the
FileDescriptorSet(prost_buildemits one viafile_descriptor_set_path) and assert every map field in the workspace emits both entry fields for a zero-valued key and value. Without that guard, adding a map to a hashed message is a silent consensus divergence. - Route
cluster/build.rsthroughpluto-build-proto.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing the Duty conversions in core/src/types.rs, consensus/src/qbft/msg.rs, and priority/src/prioritiser.rs, then inspect the map handling in core/unsigneddata.rs and cluster/build.rs. Trace hash_proto and the protobuf build configuration before designing the shared conversion and encoding helpers. Done means conversions and zero-value conventions are centralized, descriptor-driven tests cover map fields, and cluster/build.rs uses pluto-build-proto.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- blockchain, distributed-systems
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100