libp2p / libp2p/jvm-libp2p

Add support for partial messages extension

Open
#435 0 comments 0 reactions 1 assignee View on GitHub

@lucassaldanha is already working on this.

Since Feb 11, 2026.

partial-messages
Dominant language
Kotlin
Stars
366
Forks
85
Avg merge
2d 39m
Merged PRs (30d)
6

Description

## Summary

Add support for the [gossipsub partial-messages extension](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/partial-messages.md) (spec lifecycle 1A, merged via [libp2p/specs#685](https://github.com/libp2p/specs/pull/685)).

Target consumer: Teku (PeerDAS / cell-level dissemination).

## Design

Full design document lives in-repo at [`docs/partial-messages.md`](../blob/develop/docs/partial-messages.md) (lands with the first PR on this issue). Highlights:

- jvm-libp2p is a transport + per-peer bookkeeper for opaque partial-message RPCs; the client (Teku) owns encoding of `partsMetadata`, generation of `groupID`, merging, validation, reassembly, and completion detection. `partsMetadata` and `partialMessage` are fully opaque to the library.
- Client-facing API: `PartialMessagesHandler` (generic per-peer state), `publishPartial(topic, groupId, actions)`, `PublishAction(partialMessage?, partsMetadata?, nextPeerState?)`, plus a `PartialMessagesPeerFeedback` side-channel for peer scoring.
- Routing integration: suppress full-message send to partial-requesting peers, suppress IDONTWANT to peers we request partial from, replace IHAVE with `onEmitGossip` for partial-capable peers during heartbeat.
- Per-group lifecycle with TTL (in heartbeats) + DoS caps on peer-initiated groups (defaults: 255 per topic, 8 per (topic, peer)), matching go-libp2p.

## Scope

**In scope (MVP):**
- Full wire protocol: `SubOpts.requestsPartial`/`supportsSendingPartial`, `RPC.partial`, both metadata-only and payload-only variants in both directions.
- Client plug-in API (handler, publish, peer feedback).
- Routing rule integration (3 points in `GossipRouter`).
- Group lifecycle + DoS caps.

**Deferred (documented in §7 of the design doc):**
- `interop-test-client` partial-messages support.
- Partial-specific peer-scoring rules beyond the existing handshake penalties.
- Topic-level "partial-only" mode (spec explicitly defers).
- Reassembling a full `Message` back into the normal gossip flow.

## Implementation checklist

Each step is tagged with the sub-issue that tracks it. Tests are folded into each sub-issue's done-criteria; the final mixed-peer simulator scenario is tracked as a done-criterion on #456.

- [ ] 1. Per-topic `SubOpts` flag plumbing (outbound announce + inbound parse + coercion + ignore-on-unsubscribe) — #444
- [ ] 2. `PartialMessagesHandler`, `PublishAction`, `PublishActionsFn`, `GroupState` container — #446
- [ ] 3. Inbound `RPC.partial` dispatch (replace `GossipRouter.kt:476` stub with full flow) — #445
- [ ] 4. Outbound `publishPartial(...)` through `GossipRpcPartsQueue` (not bypassing) — #445
- [ ] 5. End-to-end integration test with a trivial bitmap-based handler — done-criterion on #445
- [ ] 6. Routing: full-message suppression for partial-requesting peers — #456
- [ ] 7. Routing: IDONTWANT suppression — #456
- [ ] 8. Heartbeat tick + TTL GC + DoS caps + cleanup hooks (peer disconnect, topic unsubscribe) — #449
- [ ] 9. Routing: IHAVE replacement with `onEmitGossip` — #456
- [ ] 10. Simulator scenario + mixed-peer interop test (partial + non-partial nodes on same topic) — done-criterion on #456

### Cross-cutting sub-issues

- #447 — `PartialMessagesPeerFeedback` wiring (in scope, Step 2) + partial-specific scoring rules (deferred).
- #448 — "Eager push" pattern (primarily a Teku-side concern; API must support it — verification within Step 2 / #446).

## References

- Spec: [libp2p/specs `pubsub/gossipsub/partial-messages.md`](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/partial-messages.md) (merge commit `6b6203ee`).
- Reference impl: [libp2p/go-libp2p-pubsub](https://github.com/libp2p/go-libp2p-pubsub) (`extensions.go`, `partialmessages/partialmsgs.go`, `gossipsub.go`, `pubsub.go`).
- Client reference: [OffchainLabs/prysm `prysm/partial-cells-current`](https://github.com/OffchainLabs/prysm/tree/prysm/partial-cells-current).
- Superseded earlier attempt: #433 (closed).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.