Five surfaces refuse compressed media for the same missing demuxer, and no row owns the codec brick mm-serving.md already proposes
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: -
Five surfaces across this tree refuse compressed media for the same reason -- nothing here
vendors a demuxer -- and each carries its own named residual. This issue proposes one shared
codec brick to discharge them, and records the evidence so the next reader does not
re-derive it.
The five
| Surface | What it says today |
|---|---|
src/vllm/entrypoints/openai/server_main.cpp:1538-1540 |
image: "container-format decode (PNG/JPEG -> RGB) is a named MM-SERVE residual; supply raw RGB" |
src/vllm/model_executor/models/ltx2_audio_input.cpp:43-45 |
"this project vendors no demuxer, so MP3, FLAC, OGG and MP4 audio are refused rather than misread" |
.agents/specs/ltx25-a2v-audio-input.md:465 |
"Compressed audio containers -- refused by name; no demuxer is vendored." |
src/vllm/entrypoints/openai/speech_api.cpp:193-196 |
the ENCODE side: "no mp3/opus/aac/flac encoder is vendored" |
.agents/specs/dots3-note.md ## Owed |
dots3-note W7c's container arm |
.agents/specs/mm-serving.md already proposes the right shape for this in prose: "a small
codec brick." Nothing has owned it.
Why it is not a model row's job
The surface is unbounded. libsndfile 1.2.2 reports 26 container formats, and upstream's
own fallback is PyAV/ffmpeg, which adds hundreds of demuxers
(vllm/multimodal/media/audio.py:216-258 @ 9035151d6: soundfile first, PyAV on
ImportError or a LibsndfileError whose code is in _BAD_SF_CODES).
A verified inventory of what this tree has: zero audio dependencies of any kind. No
libsndfile, no libav*/ffmpeg linkage, no libswresample, no dr_wav, stb_vorbis, minimp3,
libFLAC, opus, ogg, soxr, libsamplerate, speexdsp. third_party/ holds six entries -- blake3,
doctest, httplib, minja, nlohmann/json, Vulkan headers -- none of them audio. ffmpeg appears
only as an external process whose argv the library composes and never spawns
(src/vllm/model_executor/models/minimax_h3_mux.cpp:12).
The decision this needs
It turns on the archive contract's no-vendor rule, which is project policy rather than a
row decision. Note the precedent already exists for a deliberate exception:
docker/Dockerfile:209-215 installs the ffmpeg BINARY as a documented divergence from that
rule.
Options, roughly in ascending cost: vendor a narrow decoder set (dr_wav / minimp3 / stb_vorbis
are single-header and permissively licensed, but that is three formats, not 26); link
libsndfile optionally and refuse by name when absent; keep refusing permanently and document
raw PCM/RGB as the contract.
One record repair that should ride along
.agents/specs/ltx25-retake.md's ## Owed points at issue #924 for "A container demuxer.
Without it the audio half of retake cannot exist" -- and #924 does not resolve. That
tracking number is dangling and needs reconciling whichever way this issue is decided.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with .agents/specs/mm-serving.md and compare the five residuals in server_main.cpp, ltx2_audio_input.cpp, ltx25-a2v-audio-input.md, speech_api.cpp, and dots3-note.md. Review the cited audio fallback and Dockerfile precedent before choosing a policy option. Done means the codec brick has an owner and the dangling reference to issue #924 is reconciled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- audio-video-rtc
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100