NVIDIA / NVIDIA/TensorRT-Model-Connect

Feature: Add ornith-ai/Ornith-1.5-9B checkpoint support

Open
#1,050 2 comments 0 reactions 1 assignee View on GitHub

@roma5087 is already working on this.

Since Aug 26, 2026.

AI enhancement model-onboarding
Dominant language
Python
Stars
254
Forks
58
Avg merge
1d 7h
Merged PRs (30d)
235

Description

This issue was drafted by AI at a user's request.

Request type

New model or checkpoint support

Problem and use case

Ornith-1.5-9B is a public,
MIT-licensed, dense 9B reasoning model derived from Qwen3.5. Its documented use
cases include coding, agentic workflows, long-context chat, reasoning output,
and tool calling. The checkpoint is small enough for single-GPU deployment, but
TensorRT-Model-Connect does not currently list or validate it as a supported
checkpoint.

TensorRT-Model-Connect already has a qwen3_5 family and native runtime path
for Qwen/Qwen3.5-9B. Users should be able to build and run the Ornith
checkpoint through the same public Model Connect workflow when its architecture
is compatible, with any checkpoint-specific differences handled and validated
explicitly.

Proposed outcome

Add supported, tested ornith-ai/Ornith-1.5-9B checkpoint coverage. The initial
scope can focus on the model card's primary text-generation/chat path:

  • Build a bundle from the pinned public checkpoint through the normal trtmc
    builder interface.
  • Run the bundle through the native runtime path on a single supported GPU.
  • Reuse the existing Qwen3.5 family where the checkpoint is architecture- and
    weight-layout-compatible; record and handle any real differences rather than
    adding checkpoint-name-only behavior.
  • Preserve the checkpoint's chat-template behavior, including thinking output
    and XML tool-call serialization, within the capabilities of the public
    runtime interface.
  • Add focused upstream parity validation and list the checkpoint in the public
    supported-model documentation.

The checkpoint also contains a Qwen3.5 vision configuration and is tagged for
image-text-to-text use. If the first implementation supports only text input,
document that boundary clearly and track image/video input support separately.

Acceptance criteria
  • The exact checkpoint and an immutable revision are registered in the
    model manifest/test inventory.
  • An FP16 engine builds from the BF16 source checkpoint through the public
    Model Connect builder without manual checkpoint rewriting.
  • The resulting bundle runs through the normal native runtime on an NVIDIA
    H100 80GB for a representative chat prompt.
  • A normal reasoning/chat response and a tool-calling prompt are validated
    against the pinned upstream Transformers behavior using meaningful,
    behavior-aligned criteria.
  • The implementation records whether the existing qwen3_5 family is
    reused unchanged or which checkpoint-specific config, weights, tokenizer,
    or chat-template differences require support.
  • The supported-model documentation states the supported precision, task,
    GPU/runtime path, context-length boundary, and multimodal boundary.
  • No test threshold, passing criterion, or validator is weakened to make
    the checkpoint pass.
Model and target details
  • Model ID: ornith-ai/Ornith-1.5-9B
  • Revision: 489cb97981b8654bcfcf30ce1f94ed1b62e07b53
  • License: MIT
  • Architecture: Qwen3_5ForConditionalGeneration (model_type: qwen3_5)
  • Size/source dtype: approximately 9.65B parameters, BF16
  • Initial task: reasoning-oriented text generation/chat and tool-call
    serialization
  • Initial engine precision: FP16
  • Initial target: NVIDIA H100 80GB, single GPU
  • Expected path: public trtmc builder -> bundle -> native runtime
  • Declared context window: 262,144 tokens; initial validation may use a smaller,
    explicitly documented bounded context
Alternatives considered
  • Serve the checkpoint with vLLM, SGLang, or llama.cpp. These are documented
    upstream alternatives, but they do not produce or validate a Model Connect
    bundle for the native runtime.
  • Use the already-supported Qwen/Qwen3.5-9B checkpoint. That exercises a
    related architecture but does not provide Ornith's trained weights and
    behavior.
  • Point the existing qwen3_5 family at Ornith without adding a manifest and
    parity coverage. This would leave compatibility and regressions unsupported.
Additional context
Submission checks
  • I searched open and closed issues and found no duplicate request.
  • I removed secrets, private/internal evidence, personal paths, and
    restricted artifacts.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.