Comfy-Org / Comfy-Org/ComfyUI

[Integration] Document/register vLLM-Omni as a remote MiniMax-H3 backend

Open
#15,281 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

Hi, Comfyui developers and users!

ComfyUI now has excellent native MiniMax-H3 support through [#15224](https://github.com/Comfy-Org/ComfyUI/pull/15224). This proposal is complementary: make the vLLM-Omni custom-node integration easier to discover for users who want ComfyUI to act as a lightweight workflow client while MiniMax-H3 runs in a separate local or remote serving environment.

[vLLM-Omni PR #5756](https://github.com/vllm-project/vllm-omni/pull/5756) extends the existing `ComfyUI-vLLM-Omni` integration with:

- MiniMax-H3 T2VA through the Generate Video node
- FL2VA by connecting a first-frame image
- Ref2VA with one image plus one audio reference
- Ref2VA with one or two video references
- automatic `t2va` / `fl2va` / `ref2va` request routing
- dedicated `flow_shift` and `audio_flow_shift` controls
- an updated example workflow and integration tests

The main value compared with in-process native execution is deployment separation:

- Sageattn 2/3/CPU offload/quantization/multi-GPU parallelism
- ComfyUI can run on a creator workstation or even in CPU mode while H3 runs on a dedicated GPU server.
- ComfyUI and vLLM-Omni can use separate Python, PyTorch, and CUDA environments.
- A shared OpenAI-compatible `/v1/videos` service can be reused by ComfyUI and non-ComfyUI clients.
- The same custom-node package also connects ComfyUI to vLLM-Omni image, speech, and multimodal-understanding services.

Would the ComfyUI project be open to documenting or registering this as an external MiniMax-H3 backend, for example through the custom-node registry, an integration guide, or a community workflow/template?

### Existing Solutions

- Native MiniMax-H3 in ComfyUI: [#15224](https://github.com/Comfy-Org/ComfyUI/pull/15224), including T2VA, FL2VA, and broader Ref2VA combinations.
- MiniMax partner/API nodes for hosted generation.
- Existing vLLM-Omni custom nodes: [`apps/ComfyUI-vLLM-Omni`](https://github.com/vllm-project/vllm-omni/tree/main/apps/ComfyUI-vLLM-Omni).

The external backend is useful when the GPU server and ComfyUI host should be different machines, or when model-serving dependencies should remain isolated from ComfyUI.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing native MiniMax-H3 work in #15224 and the vLLM-Omni integration at apps/ComfyUI-vLLM-Omni, including its example workflow and integration tests. The issue does not select a repository file or documentation location; done would require a maintainer decision on an integration guide, custom-node registry entry, or community workflow/template.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
ai, backend, documentation
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.