cloudflare / cloudflare/realtime-examples

Ingest (local/buffer) websocket adapter never dials the endpoint (0 RTP)

Open
#29 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
145
Forks
29
Avg merge
2d 13h
Merged PRs (30d)
5

Description

# Bug report — Cloudflare Realtime SFU websocket adapter (ingest/local mode) never dials the endpoint

## Summary
For a `location: "local"` (ingest) websocket adapter, the SFU returns `adapterId` + a new `sessionId` from
`POST /v1/apps/{appId}/adapters/websocket/new`, but it **never opens the WebSocket to the `endpoint`** — so no
audio can be published onto the created track (0 RTP). The `location: "remote"` (egress) adapter on the same app,
same endpoint host, same auth model **does** dial correctly.

## Repro
1. Create an egress adapter (works):
`POST /v1/apps/{appId}/adapters/websocket/new` with
`{ "tracks": [{ "location":"remote","sessionId":"","trackName":"mic","endpoint":"wss://rt.wave.online/v1/realtime/agents/egress////mic?t=","outputCodec":"pcm" }] }`
→ SFU dials the endpoint, streams PCM frames (observed ~650 binary frames per short session).

2. Create an ingest adapter (does NOT dial):
`POST /v1/apps/{appId}/adapters/websocket/new` with
`{ "tracks": [{ "location":"local","trackName":"agent-voice-agent","endpoint":"wss://rt.wave.online/v1/realtime/agents/ingest////agent-voice-agent?t=","inputCodec":"pcm","mode":"buffer" }] }`
→ returns `{ tracks: [{ adapterId, trackName, sessionId (NEW), endpoint }] }` (200), but **no WebSocket
connection ever arrives at the endpoint** (our Worker logs every request; zero ingest requests observed,
while the egress endpoint on the same worker + same `wss://` base receives its dial immediately).

## Request shape vs docs
The ingest request matches the published contract exactly (`location:"local"`, `inputCodec:"pcm"`,
`mode:"buffer"`, no `sessionId` reliance — the docs say `sessionId` is ignored/auto-generated).

## Environment
- Adapter API base: `https://rtc.live.cloudflare.com/v1` (`DEFAULT_SFU_API_BASE`).
- Endpoint host: `wss://rt.wave.online` (a Cloudflare Worker on the same account).
- The endpoint is a standard WebSocket server (WebSocketPair + `acceptWebSocket`); it accepts and sends
16-bit LE 48 kHz stereo PCM in ≤32 KB chunks, matching the buffer-mode contract.

## Impact
The full voice-agent pipeline works up to TTS (STT → LLM → ElevenLabs TTS produce audio), but the agent's
reply can never be published to the room because the ingest socket never connects. This is the last blocker
for the WAVE voice-agent canary.

## Questions
1. Is there a trigger/condition for the SFU to dial the ingest endpoint that we're missing (e.g. the endpoint
must respond to a specific WS subprotocol/handshake, or the `?t=` query param is dropped on the ingest
dial specifically)?
2. Is this a known beta limitation of the ingest (local) mode?

Contributor guide

Open the contributing guide

Research direction

Reproduce the two POST /v1/apps/{appId}/adapters/websocket/new requests and compare the local/buffer and remote adapter behavior, including the returned adapterId, sessionId, endpoint, and WebSocket activity. Start with the ingest and egress request shapes described here; done means identifying the missing dial condition or confirming a beta limitation and documenting the required trigger or workaround.

Written by the indexing model from the issue text.

Assessment

Domain
backend-api-design, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.