multimodal audio in vs. transcribing an audio attachment?
- Dominant language
- Dart
- Stars
- 286
- Forks
- 94
- PR merge metrics
- No merged PRs in 30d
Description
Currently, it appears that audio is handled as a binary attachment, and is then transcribed. ref https://github.com/flutter/ai/blob/b1fbc7c8dfc6b1b0dc7c2485b4ea2f3d99befa32/lib/src/views/llm_chat_view/llm_chat_view.dart#L230
For multimodal models such as Gemini, audio as an input is natively supported.
The expectation is that instead of an audio attachment that is transcribed, the audio should be used as the input to the model directly rather than the transcription.
Contributor guide
Research direction
Start with lib/src/views/llm_chat_view/llm_chat_view.dart around line 230 and trace how an audio attachment becomes a transcription. Compare that flow with the multimodal model input path. Done means supported audio is passed directly to the model instead of being transcribed first.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- dart
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100