[Integration] Document/register vLLM-Omni as a remote MiniMax-H3 backend
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
Hi, Comfyui developers and users!
ComfyUI now has excellent native MiniMax-H3 support through [#15224](https://github.com/Comfy-Org/ComfyUI/pull/15224). This proposal is complementary: make the vLLM-Omni custom-node integration easier to discover for users who want ComfyUI to act as a lightweight workflow client while MiniMax-H3 runs in a separate local or remote serving environment.
[vLLM-Omni PR #5756](https://github.com/vllm-project/vllm-omni/pull/5756) extends the existing `ComfyUI-vLLM-Omni` integration with:
- MiniMax-H3 T2VA through the Generate Video node
- FL2VA by connecting a first-frame image
- Ref2VA with one image plus one audio reference
- Ref2VA with one or two video references
- automatic `t2va` / `fl2va` / `ref2va` request routing
- dedicated `flow_shift` and `audio_flow_shift` controls
- an updated example workflow and integration tests
The main value compared with in-process native execution is deployment separation:
- Sageattn 2/3/CPU offload/quantization/multi-GPU parallelism
- ComfyUI can run on a creator workstation or even in CPU mode while H3 runs on a dedicated GPU server.
- ComfyUI and vLLM-Omni can use separate Python, PyTorch, and CUDA environments.
- A shared OpenAI-compatible `/v1/videos` service can be reused by ComfyUI and non-ComfyUI clients.
- The same custom-node package also connects ComfyUI to vLLM-Omni image, speech, and multimodal-understanding services.
Would the ComfyUI project be open to documenting or registering this as an external MiniMax-H3 backend, for example through the custom-node registry, an integration guide, or a community workflow/template?
### Existing Solutions
- Native MiniMax-H3 in ComfyUI: [#15224](https://github.com/Comfy-Org/ComfyUI/pull/15224), including T2VA, FL2VA, and broader Ref2VA combinations.
- MiniMax partner/API nodes for hosted generation.
- Existing vLLM-Omni custom nodes: [`apps/ComfyUI-vLLM-Omni`](https://github.com/vllm-project/vllm-omni/tree/main/apps/ComfyUI-vLLM-Omni).
The external backend is useful when the GPU server and ComfyUI host should be different machines, or when model-serving dependencies should remain isolated from ComfyUI.
Contributor guide
Research direction
Start by reviewing the existing native MiniMax-H3 work in #15224 and the vLLM-Omni integration at apps/ComfyUI-vLLM-Omni, including its example workflow and integration tests. The issue does not select a repository file or documentation location; done would require a maintainer decision on an integration guide, custom-node registry entry, or community workflow/template.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- ai, backend, documentation
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100