NVIDIA / NVIDIA/TensorRT-Edge-LLM
feat: add InternVLA-N1-DualVLN vision-language navigation support
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 563
- Forks
- 135
- Avg merge
- 14h 13m
- Merged PRs (30d)
- 1
Description
Detailed description of the requested feature
Add support for InternVLA-N1-DualVLN, a dual-system vision-language navigation model:
a Qwen2.5-VL-7B planner (System 2) plus a flow-matching trajectory expert with a DINOv2
memory block (System 1), joined by a z_latents bridge. Direct export from the released
checkpoint with no repackaging step, the bridge folded into the LLM graph so the engine
emits z_latents directly, and an experimental C++ runtime that runs both systems
asynchronously in one process (System 2 plans on a background thread while System 1
samples from the newest available plan).
A working implementation and full validation (System-1 parity, latency, and 199-episode
closed-loop navigation success rate on Jetson Thor) are ready; happy to open the PR as
soon as this is approved.
Timeline
No hard deadline. Nice to have merged, not blocking anything on our side.
Describe alternatives you've considered
An existing recipe repackages the checkpoint and computes the z_latents bridge on the host
in Python. Considered keeping that approach but chose direct export + an in-graph bridge
instead, since it needed no repackaging step and let the C++ runtime own the whole forward
pass.
Target hardware/use case
Jetson Thor (sm_110, JetPack 7.1, TensorRT 10.13.3.9) for closed-loop robot navigation.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue does not name repository files, tests, or entry points. Start by reviewing the existing checkpoint-repackaging recipe and the proposed direct-export path, then inspect the stated C++ runtime and validation results. Done means direct export with an in-graph z_latents bridge, asynchronous dual-system execution, System-1 parity, latency validation, and 199-episode Jetson Thor success-rate validation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- ai, computer-vision, embedded-iot, robotics
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100