[Reasoning Engine] Google ADK BIDI streaming issue deployed on Agent engine

I'm deploying an ADK root agent onto Vertex AI Agent Engine, with the goal of supporting bidirectional audio streaming using Gemini 2.0 Flash Experimental (model="gemini-2.0-flash-exp").

While the agent responds correctly to text messages, audio messages are not recognized properly, and the agent consistently replies with "I don't understand". It appears the model isn't processing the audio input correctly or isn't returning audio output.

What I Did
> I used the official WebSocket BIDI streaming preview code from the ADK repo.
> Modified the message format to support PCM-to-WAV conversion and embedded the audio in the expected format for Vertex AI Agents.
> The model used is: gemini-2.0-flash-exp.
Observed Behavior
> Text input: Agent works fine and responds as expected.
> Audio input: After sending the audio (converted to WAV and wrapped with inline_data), the agent always replies with:
"I don't understand"
> There is no audio in the response, only this string message.

PYTHON CODE:
@app.websocket("/ws/{user_id}")
async def websocket_endpoint(websocket: WebSocket, user_id: int, is_audio: str):
await websocket.accept()
session_id, modality = await start_agent_session(str(user_id), is_audio == "true")
await agent_to_client_messaging(websocket, str(user_id), session_id, modality)

async def agent_to_client_messaging(websocket, user_id, session_id, modality):
while True:
message_json = await websocket.receive_text()
message = json.loads(message_json)
mime_type = message["mime_type"]
data = message["data"]

if mime_type == "text/plain":
user_message = data

elif mime_type == "audio/pcm":
pcm_bytes = base64.b64decode(data)
wav_bytes = pcm_to_wav(pcm_bytes)
user_message = {
"role": "user",
"parts": [
{"text": " "}, # Required placeholder
{"inline_data": {
"data": wav_bytes,
"mime_type": "audio/wav"
}}
]
}

else:
continue # Unknown mime type, skip

async for event in adk_app.async_stream_query(
message=user_message,
user_id=user_id,
session_id=session_id,
):
# handling event responses...

- I would like to know the correct approach for making this communication between the agent and the audio input possible
- I would also like to know if this is even possible currently to deploy Agent adk on agent engine and communicate with only audio-to-audio communication

从 Python WebSocket 处理程序和官方 ADK WebSocket BIDI 流式传输预览代码开始，然后跟踪对 adk_app.async_stream_query 的调用以及 audio/inline_data 负载。验证 Agent Engine 是否支持此音频到音频路径，并确定受支持的消息格式；完成的标准是确认一种可行的方法，或记录当前的限制。

由索引模型根据 Issue 内容生成。