NVIDIA / NVIDIA/TensorRT-Model-Connect
Feature: Add ornith-ai/Ornith-1.5-9B checkpoint support
@roma5087 is already working on this.
Since Aug 26, 2026.
- Dominant language
- Python
- Stars
- 254
- Forks
- 58
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 235
Description
This issue was drafted by AI at a user's request.
Request type
New model or checkpoint support
Problem and use case
Ornith-1.5-9B is a public,
MIT-licensed, dense 9B reasoning model derived from Qwen3.5. Its documented use
cases include coding, agentic workflows, long-context chat, reasoning output,
and tool calling. The checkpoint is small enough for single-GPU deployment, but
TensorRT-Model-Connect does not currently list or validate it as a supported
checkpoint.
TensorRT-Model-Connect already has a qwen3_5 family and native runtime path
for Qwen/Qwen3.5-9B. Users should be able to build and run the Ornith
checkpoint through the same public Model Connect workflow when its architecture
is compatible, with any checkpoint-specific differences handled and validated
explicitly.
Proposed outcome
Add supported, tested ornith-ai/Ornith-1.5-9B checkpoint coverage. The initial
scope can focus on the model card's primary text-generation/chat path:
- Build a bundle from the pinned public checkpoint through the normal
trtmc
builder interface. - Run the bundle through the native runtime path on a single supported GPU.
- Reuse the existing Qwen3.5 family where the checkpoint is architecture- and
weight-layout-compatible; record and handle any real differences rather than
adding checkpoint-name-only behavior. - Preserve the checkpoint's chat-template behavior, including thinking output
and XML tool-call serialization, within the capabilities of the public
runtime interface. - Add focused upstream parity validation and list the checkpoint in the public
supported-model documentation.
The checkpoint also contains a Qwen3.5 vision configuration and is tagged for
image-text-to-text use. If the first implementation supports only text input,
document that boundary clearly and track image/video input support separately.
Acceptance criteria
- The exact checkpoint and an immutable revision are registered in the
model manifest/test inventory. - An FP16 engine builds from the BF16 source checkpoint through the public
Model Connect builder without manual checkpoint rewriting. - The resulting bundle runs through the normal native runtime on an NVIDIA
H100 80GB for a representative chat prompt. - A normal reasoning/chat response and a tool-calling prompt are validated
against the pinned upstream Transformers behavior using meaningful,
behavior-aligned criteria. - The implementation records whether the existing
qwen3_5family is
reused unchanged or which checkpoint-specific config, weights, tokenizer,
or chat-template differences require support. - The supported-model documentation states the supported precision, task,
GPU/runtime path, context-length boundary, and multimodal boundary. - No test threshold, passing criterion, or validator is weakened to make
the checkpoint pass.
Model and target details
- Model ID:
ornith-ai/Ornith-1.5-9B - Revision:
489cb97981b8654bcfcf30ce1f94ed1b62e07b53 - License: MIT
- Architecture:
Qwen3_5ForConditionalGeneration(model_type: qwen3_5) - Size/source dtype: approximately 9.65B parameters, BF16
- Initial task: reasoning-oriented text generation/chat and tool-call
serialization - Initial engine precision: FP16
- Initial target: NVIDIA H100 80GB, single GPU
- Expected path: public
trtmcbuilder -> bundle -> native runtime - Declared context window: 262,144 tokens; initial validation may use a smaller,
explicitly documented bounded context
Alternatives considered
- Serve the checkpoint with vLLM, SGLang, or llama.cpp. These are documented
upstream alternatives, but they do not produce or validate a Model Connect
bundle for the native runtime. - Use the already-supported
Qwen/Qwen3.5-9Bcheckpoint. That exercises a
related architecture but does not provide Ornith's trained weights and
behavior. - Point the existing
qwen3_5family at Ornith without adding a manifest and
parity coverage. This would leave compatibility and regressions unsupported.
Additional context
- Model card: https://huggingface.co/ornith-ai/Ornith-1.5-9B
- Ornith-1.5 technical blog: https://ornith.ai/ornith_1_5.html
Submission checks
- I searched open and closed issues and found no duplicate request.
- I removed secrets, private/internal evidence, personal paths, and
restricted artifacts.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.