aws-samples / aws-samples/foundational-llm-chat
[Feature Req] Voice input support (via Amazon Transcribe?)
- Dominant language
- Python
- Stars
- 36
- Forks
- 11
- PR merge metrics
- No merged PRs in 30d
Description
I have a demo use-case with a touchscreen but no keyboard, for which it'd be useful to support audio input.
I had a working prototype of this some time ago, but would need to clean up the code before being able to propose here directly. In general, it was along the lines of:
- Use the [`amazon-transcribe` Python SDK](https://github.com/awslabs/amazon-transcribe-streaming-sdk)
- Set up the transcription client in Chainlit's `@cl.on_audio_start` hook
- `send_audio_event` in Chainlit's `@cl.on_audio_chunk` hook
- `handle_events` in Chainlit's `@cl.on_audio_end` hook (I couldn't get this working in on_audio_chunk at the time on CL v2.6.2, because both the Chainlit hook and the Transcribe handler were async generators so didn't like being nested).
Contributor guide
Research direction
Start by locating the Chainlit @cl.on_audio_start, @cl.on_audio_chunk, and @cl.on_audio_end hooks described in the issue, then review the amazon-transcribe Python SDK flow. The work is complete when audio input is transcribed through Amazon Transcribe and integrated into the chat without the async-generator issue described for Chainlit v2.6.2.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- audio-video-rtc, backend-api-design
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100