fix(cli): render video content items without image support
- Dominant language
- Rust
- Stars
- 467
- Forks
- 54
- Avg merge
- 4h 25m
- Merged PRs (30d)
- 310
Description
## Problem
Found during review of PR #1777, which aligned the CLI and server only for the image-capable case. The CLI's `apply_vlm_chat_template` builds a typed content list only when the template handles `type == 'image'` and otherwise flattens the turn to text, even though `resolve_cli_prompt` sent the request there because it carries a video. The server's raw render passes the typed list, `{"type": "video"}` item included, whenever the template handles image, video or audio content. A template that renders `video` items but has no `image` branch therefore gets the video item from the server and nothing from the CLI, and the two fronts prefill different prompts for the same request.
## Evidence
- `src/commands/generate.rs:770-772`: the early return on `!processor.supports_image_content()` runs before the per-modality `emit_video` / `emit_audio` decisions at `:798-799`; `resolve_cli_prompt` routes a video request into it at `:862`.
- `src/server/chat_request.rs:943-947`: the raw render is chosen when the template supports any of the three content types; `template_content` (`:1921-1976`, called at `:1834`) then emits a typed item for every part, `{"type": "image"}` included, whether or not the template has a branch for it.
- Audio is out of scope: the server flattens every audio-bearing request to a string with ordered sentinels (`chat_request.rs:1815-1816`, `:1929-1950`), so it never hands a template `{"type": "audio"}` items and the premise above does not apply to audio.
## Proposed fix
Decide per modality on both fronts (`supports_image_content`, `supports_video_content`) rather than gating the whole list on image support on the CLI and on any-modality support on the server. Add a CLI/server parity test with a synthetic template that has a `video` branch and no `image` branch, following `cli_video_fallback_renders_the_server_prompt_for_an_imageless_template` (`src/commands/generate_tests.rs:1305`) and its server twin `video_frames_render_for_an_imageless_template_is_the_flattened_turn` (`src/server/chat_request_tests.rs:3822`).
## Acceptance criteria
- [ ] For such a template and a request with one native video and a question, the CLI and the server render byte-identical prompts (test).
- [ ] Existing image-capable and imageless parity tests still pass unchanged.
## Verification
```bash
cargo test --workspace --profile test-fast --features metal,accelerate
cargo clippy --workspace --all-targets -- -D warnings
cargo fmt --all -- --check
```
The new parity test must fail with the fix reverted.
Contributor guide
Research direction
Start with src/commands/generate.rs:770-862 and src/server/chat_request.rs:1815-1976 to trace the modality decisions and raw rendering. Read the parity tests at src/commands/generate_tests.rs:1305 and src/server/chat_request_tests.rs:3822, then add the requested synthetic video-only template case. Done means the CLI and server prompts are byte-identical and the existing parity tests plus the listed cargo checks pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100