NVIDIA / NVIDIA/TensorRT-Edge-LLM

feat: add InternVLA-N1-DualVLN vision-language navigation support

Open
#190 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

New Model
Dominant language
Python
Stars
563
Forks
135
Avg merge
14h 13m
Merged PRs (30d)
1

Description

Detailed description of the requested feature

Add support for InternVLA-N1-DualVLN, a dual-system vision-language navigation model:
a Qwen2.5-VL-7B planner (System 2) plus a flow-matching trajectory expert with a DINOv2
memory block (System 1), joined by a z_latents bridge. Direct export from the released
checkpoint with no repackaging step, the bridge folded into the LLM graph so the engine
emits z_latents directly, and an experimental C++ runtime that runs both systems
asynchronously in one process (System 2 plans on a background thread while System 1
samples from the newest available plan).

A working implementation and full validation (System-1 parity, latency, and 199-episode
closed-loop navigation success rate on Jetson Thor) are ready; happy to open the PR as
soon as this is approved.

Timeline

No hard deadline. Nice to have merged, not blocking anything on our side.

Describe alternatives you've considered

An existing recipe repackages the checkpoint and computes the z_latents bridge on the host
in Python. Considered keeping that approach but chose direct export + an in-graph bridge
instead, since it needed no repackaging step and let the C++ runtime own the whole forward
pass.

Target hardware/use case

Jetson Thor (sm_110, JetPack 7.1, TensorRT 10.13.3.9) for closed-loop robot navigation.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue does not name repository files, tests, or entry points. Start by reviewing the existing checkpoint-repackaging recipe and the proposed direct-export path, then inspect the stated C++ runtime and validation results. Done means direct export with an in-graph z_latents bridge, asynchronous dual-system execution, System-1 parity, latency validation, and 199-episode Jetson Thor success-rate validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
ai, computer-vision, embedded-iot, robotics
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.