paritytech / paritytech/host-rust-core
RFC: Host-mediated realtime media sessions
Nobody has claimed this yet.
- Dominant language
- Swift
- Stars
- 10
- Forks
- 3
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 148
Description
Source RFC: #839 · docs/rfcs/realtime-media-sessions.md
Core implementation: to follow
Summary
A Media service that lets a product run an audio or video call, with screen
sharing, without implementing, embedding, or observing any realtime transport.
The Host owns the connections, the capture devices, the codecs, the audio route,
and the pictures on screen. The product carries opaque signalling over a channel
it already has, says where each participant's picture belongs, and is told how
the call is going.
This is the tracking issue for the implementation. The first implementation is
WebRTC, with the Host running the peer connections. Nothing a product sees says
so: it handles sealed messages and Host-minted handles, not transport detail.
Requirements
- No media reaches a product: no frames, no tracks, no device handles.
- Signalling belongs to the Host: it offers, answers and exchanges candidates
over the statement store, and a product neither carries nor sees any of it. No
session detail and no participant's address reaches product code, and nothing
reports bitrates, round-trip times, candidates, or addresses. The Host owns that subscription, so calling needs no
statement permission of its own. - A call may be audio only. Camera and screen sharing are optional per
participant and per direction. - A call needs no visible surface: audio connects while the product is still
opening, so a background worker can answer. - The Host draws each incoming picture — a participant's camera or their shared
screen — into rectangles the product places, at a depth the product chooses.
Rectangles are the contract for every product; an element-based attach helper
is SDK sugar over them. - Screen capture is not a device permission: the Host runs its own picker, the
user chooses what is shared, and the product never names a window or a
display. - Camera and microphone use the existing device permissions.
- Devices belong to the Host: which microphone and camera are used, gain, echo
cancellation, routing, and the in-call affordance for switching them. A
product states what it wants sent and may express a preference, such as the
front-facing camera or a speakerphone-style call; the Host or the OS may
override it, and the product is told what is actually live and what kind of
device the Host chose, so its call UI can show the truth. A product never
learns a device name, model, or list. - Calling is its own permission, asked once and remembered, not the existing
WebRTC grant whose load-time resolution is browser-specific. A Host may offer
to allow a single call instead, which is prompt behaviour rather than a second
kind of grant. - A remembered grant cannot become an invisible call: the Host shows an in-call
indicator a product cannot suppress and the user can end the call from. - A call ends when the user withdraws camera or microphone access, or revokes
the calling permission. - Participants are Host-minted handles. The Host is never told a product's own
contact identities. - Whether a call connects directly or through a relay is the Host's choice and
invisible to the product. - A Host that cannot provide the whole stack reports the service as unsupported
rather than working partially. - Calls between more than two people are allowed. A Host must carry at least
six participants, which needs no new server; beyond that it may refuse, and
the refusal is reported so a product can grey out the control. Larger calls
would need infrastructure this work does not cover. - Recording and product-chosen codecs are out of scope, as is waking a product
that is not running for an incoming call. - The product keeps what it already owns: who may call whom, peer identity,
ringing and decline, and call history.
Core implementation scope
Mediaservice definition, versioned types, and wire ids.- Dispatcher and generated product clients.
- Session lifecycle, participants joining and leaving, ownership, consent, and
teardown on permission withdrawal. - Host-owned signalling over the statement store.
- Host WebRTC binding: peer connections, capture including the screen picker,
echo cancellation, audio session, connectivity with Host-minted relay
credentials, and compositing. - Conformance fixtures for the privacy guarantees, not only a working call.
- Reference product flow: invite, accept, end.
Implementation references
- RFC document: #839
- Background delivery to a product that is not running: handled separately
Target products
- DIM2, as an SPA
- Chat, as an SPA
- T3ams, for audio and video calling with screen sharing
- Meet, including screen sharing
- any other product needing peer-to-peer audio, video, or sharing
Tasks
- RFC document body — #839
- RFC review and acceptance
-
Mediaservice definition, versioned types, wire ids - Dispatcher and generated clients
- Session lifecycle, ownership, consent
- Host-owned signalling and incoming-call delivery
- Compositing contract for cameras and shared screens, including clamping
rules - Device ownership and live-state reporting contract
- Calling permission, including revocation ending a live call
- Privacy conformance fixtures
- Product client wrappers
- Reference flow: invite, accept, end
- Decide where the WebRTC binding lives: a trait each Host implements, or
shared Host code - Decide whether a background worker may start an audio-only call before
the product's surface is open - Host adoption
- Epoca
- dotli-community
- iOS
- Android
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with docs/rfcs/realtime-media-sessions.md and the RFC review and acceptance task from #839. Map the listed service, dispatcher, lifecycle, signalling, WebRTC, privacy fixture, and reference-flow tasks, including the unresolved implementation choices. Done means the accepted contract is implemented across the listed scope and its privacy guarantees and invite, accept, and end flow are covered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- api, audio-video-rtc, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100