cloudflare / cloudflare/agents
Voice SDK: add native full-duplex provider and AI SDK tool lifecycle
- Dominant language
- TypeScript
- Stars
- 5.6k
- Forks
- 711
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 53
Description
**Describe the gap**
@cloudflare/voice 0.3.5 exposes a Transcriber session for continuous STT and withVoice owns the cascaded onTurn plus TTS pipeline. A full-duplex speech provider such as Grok Voice, Gemini Live, or OpenAI Realtime emits user transcripts, assistant audio, interruptions, and function calls on one provider session.
There is no native full-duplex provider contract and no place to attach an AI SDK ToolSet to that session. Consumers must replace the transcriber role with a custom session adapter and implement function-call validation, execution ordering, result submission, and close settlement themselves.
**To reproduce**
1. Extend withVoice(Agent).
2. Connect a full-duplex realtime provider through createTranscriber.
3. Register the same AI SDK tools that another Agent surface uses.
4. Let the provider emit a function call while an earlier function call is still executing.
5. The Voice interfaces have no tool registry, execution lifecycle, serialization or drain contract, or function-result submission hook.
**Expected behavior**
Please add a native full-duplex provider session surface that:
- owns bidirectional audio, transcripts, playback interruption, and provider session close;
- accepts an AI SDK ToolSet so schemas, validation, and handlers remain canonical;
- serializes function calls or defines explicit drain semantics when provider events overlap;
- returns function results through the provider session before the next response;
- settles pending final transcripts and tool results before close; and
- permits first-party provider adapters without consumers replacing Voice lifecycle behavior.
This would let applications keep withVoice as the call owner and keep provider code as a narrow adapter instead of building a parallel voice and tool lifecycle.
**Screenshots**
Not applicable.
**Version**
- @cloudflare/voice: 0.3.5
- agents: 0.20.1
- ai: 7.0.58
**Additional context**
The existing Transcriber and TranscriberSession types are a good continuous-STT contract. The missing surface is the provider-native speech-to-speech and function-tool equivalent.
Contributor guide
Research direction
Read the existing Transcriber and TranscriberSession types, then inspect the withVoice and createTranscriber entry points. Define the missing full-duplex session boundary and ToolSet lifecycle, including overlapping calls, result submission, interruptions, and close settlement; done means first-party adapters can use it without replacing Voice lifecycle behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai, audio-video-rtc, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100