ObolNetwork / ObolNetwork/charon
dkg/pedersen: bundles arriving before protocol start are dropped and never redelivered
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 222
- Forks
- 138
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 35
Description
Problem
During a replace-operator ceremony (also applies to any pedersen reshare/DKG), a node that starts the protocol a few seconds later than its peers fails the ceremony:
07:31:21.901 ERRO pedersen Dropping deal bundle, context done {"from": "16Uiu..."}
07:31:23.144 ERRO pedersen Dropping deal bundle, context done {"from": "16Uiu..."}
07:31:23.340 ERRO pedersen Dropping deal bundle, context done {"from": "16Uiu..."}
07:31:23.932 INFO cmd Starting pedersen reshare...
07:31:33.944 WARN pedersen Sending DKG complaints, deals from these dealers failed verification {"dealer_indices": "[0 1 5]"}
07:31:53.937 ERRO pedersen [dkg-log Public polynomial missing - evicting dealer 5]
07:31:53.937 ERRO pedersen [dkg-log Public polynomial missing - evicting dealer 1]
07:31:53.937 ERRO pedersen [dkg-log Public polynomial missing - evicting dealer 0]
07:31:53.940 ERRO cmd Application failed to start: ... process-justifications: only 3/5 valid deals - dkg abort
Root cause
Peers ahead in the ceremony broadcast their deal bundles before this node's kyber protocol starts reading Board.IncomingDeal(). The p2p handler pushes into an unbuffered channel (dkg/pedersen/board.go), blocks until the request context expires (5s defaultRcvTimeout), and drops the bundle. The drop is permanent:
Sender.SendAsyncis fire-and-forget, and the handler acks success even after dropping, so the sender never retransmits.- On
main, the dedup map records the bundle signature before delivery, so even a retransmit would be refused as a duplicate.
The missing deals then cascade deterministically: complaints against the affected dealers → their justifications cannot be verified without their public polynomials (which were in the dropped deal bundles) → dealers evicted → fewer valid deals than oldThreshold → dkg abort.
Affected versions
Verified present on main and v1.10.3 (the unbuffered-channel drop with success ack is identical; v1.10.3 predates the dedup map but the sender never retransmits anyway).
Fix
Buffer the deal/response/justification channels to cluster size so bundles arriving before the protocol starts reading are queued instead of dropped, and un-record dropped bundles from the dedup map so a redelivery is accepted.
Workaround
Retry the ceremony with all operators starting as close to simultaneously as possible, and ensure no node is resource-starved (the failing node was ~5-8s behind its peers).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in dkg/pedersen/board.go and trace Board.IncomingDeal(), the p2p handler, Sender.SendAsync, and the dedup map around defaultRcvTimeout. Buffer deal, response, and justification channels to cluster size, ensure dropped bundles can be accepted on redelivery, and verify that bundles arriving before protocol start are not permanently lost.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- blockchain, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 67/100