trtllm-serve broken with 0.20.0 release

System Info

This is an async overview (not sure how it passed internal tests?), but it seems that trtllm-serve fails when handling multimodal data; the user would see the failing cause relates to applying chat templates:

/opt/conda/envs/foo/lib/python3.12/site-packages/fastapi/routing.py:212: RuntimeWarning: coroutine 'AsyncMultimodalDataTracker.retrieve_all_mm_data' was never awaited
Coroutine created at (most recent call last)
File "/opt/conda/envs/foo/lib/python3.12/site-packages/starlette/routing.py", line 715, in call
await self.middleware_stack(scope, receive, send)
File "/opt/conda/envs/foo/lib/python3.12/site-packages/starlette/routing.py", line 735, in app
await route.handle(scope, receive, send)
File "/opt/conda/envs/foo/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
await self.app(scope, receive, send)
File "/opt/conda/envs/foo/lib/python3.12/site-packages/starlette/routing.py", line 76, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/opt/conda/envs/foo/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/opt/conda/envs/foo/lib/python3.12/site-packages/starlette/routing.py", line 73, in app
response = await f(request)
File "/opt/conda/envs/foo/lib/python3.12/site-packages/fastapi/routing.py", line 301, in app
raw_response = await run_endpoint_function(
File "/opt/conda/envs/foo/lib/python3.12/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
return await dependant.call(**values)
File "/opt/conda/envs/foo/lib/python3.12/site-packages/tensorrt_llm/serve/openai_server.py", line 186, in openai_chat
conversation, mm_coroutines = parse_chat_messages_coroutines(request.messages, self.model_config)
File "/opt/conda/envs/foo/lib/python3.12/site-packages/tensorrt_llm/serve/chat_utils.py", line 239, in parse_chat_messages_coroutines
return conversation, mm_data_tracker.retrieve_all_mm_data()
return await dependant.call(**values)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

The error for the user is:

{"object":"error","message":"No chat template found for the given tokenizer and tools.","type":"BadRequestError","param":null,"code":400}

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Compile an engine file and run trtllm-serve . in the folder with the engine file.

Expected behavior

Requests should work without failing.

actual behavior

Requests fail due to async mismangement.

additional notes

--