aws-samples / aws-samples/foundational-llm-chat

[Feature Req] Voice input support (via Amazon Transcribe?)

Open
#17 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
36
Forks
11
PR merge metrics
No merged PRs in 30d

Description

I have a demo use-case with a touchscreen but no keyboard, for which it'd be useful to support audio input.

I had a working prototype of this some time ago, but would need to clean up the code before being able to propose here directly. In general, it was along the lines of:
- Use the [`amazon-transcribe` Python SDK](https://github.com/awslabs/amazon-transcribe-streaming-sdk)
- Set up the transcription client in Chainlit's `@cl.on_audio_start` hook
- `send_audio_event` in Chainlit's `@cl.on_audio_chunk` hook
- `handle_events` in Chainlit's `@cl.on_audio_end` hook (I couldn't get this working in on_audio_chunk at the time on CL v2.6.2, because both the Chainlit hook and the Transcribe handler were async generators so didn't like being nested).

Contributor guide

Open the contributing guide

Research direction

Start by locating the Chainlit @cl.on_audio_start, @cl.on_audio_chunk, and @cl.on_audio_end hooks described in the issue, then review the amazon-transcribe Python SDK flow. The work is complete when audio input is transcribed through Amazon Transcribe and integrated into the chat without the async-generator issue described for Chainlit v2.6.2.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
audio-video-rtc, backend-api-design
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.