livepeer / livepeer/go-livepeer

BYOC protocol cleanups (from SDK implementation)

Open
#3,913 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: triage
Dominant language
Go
Stars
586
Forks
226
Avg merge
1d 17h
Merged PRs (30d)
19

Description

Six rough edges in BYOC's wire surface, surfaced while implementing a
Python pipeline SDK against the existing protocol. None blocking;
flagging for backlog.

Protocol cleanups

  1. Inconsistent route prefixes. Same operation appears as /process/stream/* (gateway), /ai/stream/* (orch), /stream/* (runner). Unify or rename — the /ai/ prefix is historical baggage from when AI workers were a separate path.

  2. Trickle control channel bypass for /update is a permanent workaround. byoc/stream_gateway.go has a commented-out controlPub := stream.ControlPub with a "broke on base64 binary" comment. Either fix trickle's binary handling or formalize HTTP as the control plane and remove the trickle control channel from the protocol.

  3. /ai/stream/payment leaks payment concern into the protocol surface. Separate endpoint just for payment forwarding feels misplaced. Bundle into existing requests (header-based, like Livepeer-Balance) or document as orch-internal so SDK authors don't think they need to model it.

  4. Capability registration as an out-of-band dance. Every BYOC compose ships a register_capability sidecar that POSTs to the orchestrator with a separate auth scheme (orchSecret Bearer). Self-registration from the runner with heartbeats would be cleaner — no sidecar, no orchestrator-must-be-up-at-runner-startup constraint.

  5. No explicit backpressure semantics in trickle. Publisher writes, subscriber reads; a slow subscriber just causes queue/drop with no protocol signal back to the publisher ("slow down" / "buffer full"). Critical for real-time A/V quality at scale.

  6. Overlapping BYOC route prefixes. /process/request/* (batch BYOC), /process/stream/* (live BYOC), /ai/stream/* (orch internal) — three operations sharing the BYOC namespace via two different prefixes. Worth considering more orthogonal naming for a v2 protocol. (Old non-BYOC batch path is deprecated and ignored.)

Implementation polish (small fixes)

Smaller code-level cleanups uncovered while implementing the SDK against the existing wire protocol. Filed as separate issues so they can be picked up independently:

  • #3922 — BYOC gateway: ffmpeg output process logs spurious "Broken pipe" / "Conversion failed!" on every /stream/stop. Same pattern duplicated in server/ai_live_video.go. See issue for two proposed approaches (low-risk noise suppression vs. shutdown-sequencing fix).

Context: implementation experience from livepeer/livepeer-python-gateway#8. Happy to break out any of the protocol items above into individual issues if useful.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Begin by selecting one of the six protocol items; the issue names byoc/stream_gateway.go for the /update workaround and server/ai_live_video.go for related shutdown logging. No test files or single entry point are identified, so done requires a separately scoped change with explicit protocol behavior and validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend-api-design, networking
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.