block / block/buzz

Voice notes: agents receive only an mp4 link, no transcript or transcription path

Open
#7,339 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

## Summary

As of 0.5.22, when a human sends a voice note to an agent (Buzz bot / DM), the agent receives only a media file link — e.g. `voice-note-*.mp4`, with `imeta` tags for duration/size — and no text transcript. Agents have no built-in audio transcription or playback capability, so they cannot actually read what was said in a voice note addressed to them. Only a human on the receiving end, who can play the file back, gets the content.

This makes voice notes a dead end when the recipient is an agent: the event arrives, but the substance of the message is unreadable to the thing it was sent to.

## Repro

1. Send a voice note to an agent in a DM or channel (tested 2026-09-04, v0.5.22).
2. Agent's event payload contains only the mp4 link and `imeta` metadata (duration, size) — no transcript field.
3. Agent has no tool to transcribe or otherwise access the audio content, so it cannot respond to what was said.

## Ask

Either of these would close the gap:

- **(a)** Auto-attach a text transcript to voice-note events, so agents that read events already get the transcript alongside the media link, or
- **(b)** Expose an accessible transcription endpoint/tool that agents can call against the media URL on demand.

Either unblocks agents from participating in voice-note conversations rather than only seeing an opaque file link.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the voice-note event payload for agent DMs and channels, including the mp4 media link and imeta metadata described in the reproduction. Compare the existing event path with the two proposed approaches: attaching a transcript or exposing a transcription tool or endpoint. Done means an agent can obtain and read the voice note's text while retaining access to the media.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, audio-video-rtc, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.