TEN-framework / TEN-framework/ten-framework

[FEATURE] Enable ASR translation outputs for ASR vendors that support real-time translation

Open
#1,532 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
11.1k
Forks
1.4k
Avg merge
2d 15m
Merged PRs (30d)
22

Description

Background

Several ASR vendors (including Soniox and Speechmatics) offer real-time translation capabilities, but current ASR extensions don't expose translation outputs through their configuration parameters. Additionally, the standard ASR API in ten_ai_base lacks any specification for handling translation results. Supporting translation in ASR extensions would enable valuable use cases like live translation.

API

The standard ASR API in ten_ai_base currently provides the asr_result Data output. To support translation, we propose adding an optional asr_translation_result Data output with identical properties to asr_result. These two outputs operate independently—each maintains its own audio alignment via start_ms and duration_ms fields, without requiring synchronization between transcription and translation results.

Implementation

Decoupling asr_result from asr_translation_result significantly simplifies implementation across different vendor APIs, which handle translation differently:

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the standard ASR API in ten_ai_base and inspect the existing asr_result Data output and the ASR extensions for Soniox and Speechmatics. Compare how each vendor represents transcription and translation timing. Done means an optional asr_translation_result output is specified with independent audio alignment and can represent the supported vendor translation results.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, api, audio-video-rtc
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.