未来是否可以把语音交互一起并入到视频交互模型中去？

目前的模型是基于视觉交互的，语音是以旁路的形式接入的。准确地说，是ASR（语音转文字）送入到模型中去，然后模型的输出文本，再TTS（文字转语音）。所以语音不是交互的。
迫切希望未来可以视觉交互+语音交互。就是说，视频信息、语音信息，全部汇集到一个模型中去决策、调度。
并且视觉交互如果支持多路视频流，那就更好了！

No files, tests, or entry points are identified. Start by mapping the existing visual interaction path and the ASR and TTS side path described in the issue, then determine how video, audio, and multiple video streams would be coordinated. Done would require an agreed implementation scope and a working unified interaction model, rather than a small isolated change.

Written by the indexing model from the issue text.