Voice notes: agents receive only an mp4 link, no transcript or transcription path
- Dominant language
- Rust
- Stars
- 32.7k
- Forks
- 4.3k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 253
Description
## Summary
As of 0.5.22, when a human sends a voice note to an agent (Buzz bot / DM), the agent receives only a media file link — e.g. `voice-note-*.mp4`, with `imeta` tags for duration/size — and no text transcript. Agents have no built-in audio transcription or playback capability, so they cannot actually read what was said in a voice note addressed to them. Only a human on the receiving end, who can play the file back, gets the content.
This makes voice notes a dead end when the recipient is an agent: the event arrives, but the substance of the message is unreadable to the thing it was sent to.
## Repro
1. Send a voice note to an agent in a DM or channel (tested 2026-09-04, v0.5.22).
2. Agent's event payload contains only the mp4 link and `imeta` metadata (duration, size) — no transcript field.
3. Agent has no tool to transcribe or otherwise access the audio content, so it cannot respond to what was said.
## Ask
Either of these would close the gap:
- **(a)** Auto-attach a text transcript to voice-note events, so agents that read events already get the transcript alongside the media link, or
- **(b)** Expose an accessible transcription endpoint/tool that agents can call against the media URL on demand.
Either unblocks agents from participating in voice-note conversations rather than only seeing an opaque file link.
Contributor guide
Research direction
Start by tracing the voice-note event payload for agent DMs and channels, including the mp4 media link and imeta metadata described in the reproduction. Compare the existing event path with the two proposed approaches: attaching a transcript or exposing a transcription tool or endpoint. Done means an agent can obtain and read the voice note's text while retaining access to the media.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, audio-video-rtc, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100