Support audio in multimodal messages
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 589
- Forks
- 76
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 3
Description
Is your feature request related to a problem? Please describe.
LLM APIs have started supporting audio input, so it would be beneficial for RAIMultimodalMessages to support audio as well.
Describe the solution you'd like
MultimodalMessage class (https://github.com/RobotecAI/rai/blob/5d3a8f33f20e6ccfebf4fecceb3ef7d2bc70d0d1/src/rai/rai/messages/multimodal.py#L38) should support audio input.
Describe alternatives you've considered
This is the only suitable solution within the current architecture.
Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/rai/rai/messages/multimodal.py at the MultimodalMessage class referenced in the issue, then inspect how existing multimodal inputs are represented and validated. Determine the project’s expected audio input shape and add support so audio can be passed through the current architecture consistently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100