[Bug] trtllm-serve /v1/chat/completions: audio_url with data: URI base64 fails with "Supplied filename too long", only image_url decodes data: scheme

System Info

• TensorRT-LLM version: 1.2.0rc6
• Container image: nvcr.io/nvidia/tensorrt-llm/release:1.2.0rc6
• Model: nvidia/Phi-4-multimodal-instruct-NVFP4 (FP4 weights from NVIDIA)
• Endpoint: POST /v1/chat/completions
• GPU: NVIDIA RTX 5080 (Blackwell, sm_120), single GPU
• CUDA / Driver: as shipped in the official image
• Host OS: Windows 11 with Docker Desktop on WSL2 (container is Linux)
• Client: curl from the host, also reproduced from Python httpx

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Start the server (single-GPU, default settings, no source patches):

docker run --gpus all --rm -p 8000:8000 --ipc=host \
  -e HF_HOME=/root/.cache/huggingface \
  -v $HOME/.cache/huggingface:/root/.cache/huggingface \
  nvcr.io/nvidia/tensorrt-llm/release:1.2.0rc6 \
  trtllm-serve nvidia/Phi-4-multimodal-instruct-NVFP4 \
    --host 0.0.0.0 --port 8000 \
    --trust_remote_code \
    --max_seq_len 8192 \
    --kv_cache_free_gpu_memory_fraction 0.5

Wait for the server to finish loading the model (a few minutes for Phi-4-MM the first time — log will show INFO: Started server process and Application startup complete.). Then send a chat-completions request with audio_url carrying an inline data:audio/wav;base64,... URI (here a ~16 s mono 16 kHz PCM WAV, ~500 KB → ~670 KB after base64):

B64=$(base64 -w 0 sample.wav)
curl -sS -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"nvidia/Phi-4-multimodal-instruct-NVFP4\",
    \"messages\": [{
      \"role\": \"user\",
      \"content\": [
        {\"type\": \"audio_url\", \"audio_url\": {\"url\": \"data:audio/wav;base64,${B64}\"}}
      ]
    }],
    \"max_tokens\": 512,
    \"temperature\": 0
  }"

The same request shape with data:image/...;base64,... works fine for image_url on other multimodal models.

Expected behavior

audio_url with a data:audio/<mime>;base64,... URI should be base64-decoded into raw audio bytes, then handed off to the model's audio processor — exactly mirroring how image_url already handles data:image/<mime>;base64,... URIs in chat-completions today.

A successful response should look like:

{
  "id": "chatcmpl-...",
  "choices": [{
    "index": 0,
    "message": {"role": "assistant", "content": "<transcribed text>"},
    "finish_reason": "stop"
  }],
  "usage": {...}
}

actual behavior

The request returns HTTP 400:

{
  "object": "error",
  "message": "cannot unpack non-iterable NoneType object",
  "type": "BadRequestError",
  "param": null,
  "code": 400
}

The server log shows libsndfile complaining that the data URI is a too-long filename, followed by audios=None reaching the multimodal preprocessor:

QBhAHQAewCGAJMApADDANYA6QD9AB8B...': Error : Supplied filename too long.
[TRT-LLM] [W] Basic input processor failed: cannot unpack non-iterable NoneType object.
[TRT-LLM] [E] Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/serve/openai_server.py", line 574, in openai_chat
    promise = self.llm.generate_async(
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/llmapi/llm.py", line 457, in generate_async
    prompt_token_ids, extra_processed_inputs = input_processor_with_hash(
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/inputs/registry.py", line 738, in input_processor_wrapper
    return input_processor(inputs, sampling_params)
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/_torch/models/modeling_phi4mm.py", line 915, in __call__
    audio_inputs = self.processor.audio_processor(
  File "/root/.cache/huggingface/modules/transformers_modules/.../processing_phi4mm.py", line 355, in __call__
    for audio_data, sample_rate in audios:
TypeError: cannot unpack non-iterable NoneType object

So:

• libsndfile receives the full data:audio/wav;base64,<thousands of chars> string and tries to treat it as a filesystem path (POSIX ENAMETOOLONG, >255 bytes), returns nothing.
• The audio loader returns None.
• The Phi-4-MM processor then crashes when iterating audios.

The same workflow with data:image/...;base64,... on image_url works correctly on other multimodal models — only audio is broken.

additional notes

Root cause (from source inspection)

In tensorrt_llm/inputs/utils.py, async_load_audio only branches on three URL schemes:

if parsed_url.scheme in ["http", "https"]:
    resp = _safe_request_get(audio, stream=False)
    audio = BytesIO(resp.content)
elif parsed_url.scheme in ("", "file"):
    audio = _normalize_file_uri(audio)
else:
    raise ValueError(f"Unsupported URL scheme: {parsed_url.scheme!r}")

For a data: URI, urllib.parse.urlparse(...).scheme == "data". There is an is_base64 parameter on the same loader that would base64-decode the body, but it is never set to True for data: URIs — the caller in tensorrt_llm/serve/chat_utils.py:parse_chat_message_content_part (audio_url branch) simply passes the URL through.

In practice the request seems to reach the soundfile/libav-backed loader before the URL-scheme check fires (or the scheme check is silently swallowed elsewhere), which is why the user-facing error is Supplied filename too long from libsndfile rather than Unsupported URL scheme.

By contrast, the image_url branch in the same chat-completions parser auto-detects data:image/... and decodes it (the standard image_b64 path documented in examples/openai_chat_client_for_multimodal). The asymmetry between image_url (works) and audio_url (silently fails) is the bug.

Suggested fix

In the audio_url branch of parse_chat_message_content_part (tensorrt_llm/serve/chat_utils.py), mirror the existing image_url logic — detect url.startswith("data:"), split off the base64 payload, and call async_load_audio with is_base64=True. Pseudo-diff:

if part_type == "audio_url":
    url = part["audio_url"]["url"]
    audio_kwargs = resolve_media_io_kwargs(
        mm_data_tracker._multimodal_server_config,
        mm_data_tracker.request_media_io_kwargs, "audio",
    )
    if url.startswith("data:"):
        # data:audio/<mime>;base64,<payload>  →  decode inline
        _, payload = url.split(",", 1)
        audio_kwargs["is_base64"] = True
        data = async_load_audio(payload, **audio_kwargs)
    else:
        # existing http/https/file branch
        data = async_load_audio(url, **audio_kwargs)
    return MultimodalData(modality="audio", data=data, is_embedding=False)

Workarounds for users hitting this now

Either is acceptable until the fix lands:

1. Host audio over HTTP and pass an http://... URL to audio_url (TRT-LLM fetches and decodes it).
2. Bind-mount a host directory into the container and pass file:///path/to/audio.wav (TRT-LLM reads the file directly).

Both bypass the data-URI path entirely.

Why this matters

OpenAI-compatible clients (LangChain, llama-index, custom eval pipelines) often send audio as inline base64 — it removes the dependency on an external file server and works the same way as image_url. The asymmetry between image_url (works) and audio_url (silently fails) is a footgun for anyone setting up multimodal eval against trtllm-serve, especially because the user-facing error message ("cannot unpack non-iterable NoneType object") doesn't hint at the actual cause ("the data URI was never decoded").

I'm happy to put together a PR with the snippet above plus a regression test if that helps.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

Start by reproducing the request, then inspect parse_chat_message_content_part in tensorrt_llm/serve/chat_utils.py and async_load_audio in tensorrt_llm/inputs/utils.py, comparing the audio_url path with image_url handling. Add a regression test for an inline data:audio/...;base64 URI and verify that the request decodes audio successfully instead of producing the filename-too-long or NoneType errors.