NVIDIA / NVIDIA/TensorRT-Model-Connect

Feature: Add Qwen3-VL-8B-Instruct to the qwen_vl family

Open
#1,388 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
254
Forks
58
Avg merge
1d 8h
Merged PRs (30d)
223

Description

Summary

Extend the qwen_vl family's existing Qwen/Qwen3-VL-2B-Instruct support to
Qwen/Qwen3-VL-8B-Instruct.

Why

Both checkpoints share the same architecture class
(Qwen3VLForConditionalGeneration, model_type: qwen3_vl) and only differ in
scale:

2B (supported) 8B (this issue)
Text layers / hidden / heads 28 / 2048 / 16 36 / 4096 / 32
Vision depth / hidden 24 / 1024 27 / 1152

families/qwen_vl/config.py reads these dimensions from the checkpoint's own
config.json rather than hardcoding the 2B shape, so this is expected to be a
scale-up within the existing family rather than a new architecture.

Apache-2.0, ~19.7M downloads on Hugging Face.

Plan

  • Add a manifest + test case for Qwen/Qwen3-VL-8B-Instruct under
    families/qwen_vl/tests/manifests/.
  • Build and run the real GPU E2E (native generate / VL-inference CLI path)
    against the pinned checkpoint revision.
  • If the 8B checkpoint needs tensor-parallel to fit comfortably, evaluate
    qwen_vl's existing TP builder path (already used for
    Qwen/Qwen2.5-VL-3B-Instruct).
  • No shared-file changes expected (no new task type, no new CLI surface) —
    should stay entirely inside families/qwen_vl/.

Non-goals

Quantized precisions, additional Qwen3-VL sizes beyond 8B, streaming
generation.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in families/qwen_vl/config.py and inspect the existing manifest and tests under families/qwen_vl/tests/manifests/, especially the Qwen3-VL-2B-Instruct case. Add the 8B manifest and test, then build and run the pinned-checkpoint GPU E2E through the native generate or VL-inference CLI path. If it does not fit comfortably, evaluate the existing tensor-parallel builder path; done means the real E2E passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.