NVIDIA / NVIDIA/TensorRT-Edge-LLM
Qwen3.8-27B NVFP4 base engine build fails: Myelin removeEmptyProducersFromSubgraph not implemented for PLUGIN_V3 (Jetson Thor, TRT 10.13.3.9 and 10.14.1.48)
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 563
- Forks
- 135
- Avg merge
- 14h 13m
- Merged PRs (30d)
- 1
Description
Environment
- Jetson AGX Thor (sm_110), L4T R38.4 (JetPack 7.1-class), CUDA 13.0
- TensorRT-Edge-LLM 0.10.0 (bb29145)
- TensorRT tested: 10.13.3.9-1+cuda13.0 (JetPack repo) AND 10.14.1.48-1+cuda13.0 (sbsa CUDA network repo) — same failure on both
Repro
Checkpoint: NVFP4 Qwen3.8-27B (ModelOpt NVFP4, group_size 16, MTP extra tensors), e.g. https://huggingface.co/lyf/Qwen3.8-27B-Heretic-ARA-NVFP4-MTP-VL
tensorrt-edgellm-export CKPT OUT --mtp→ succeeds (after locally patching two exporter issues: (a) system scipy 1.8 vs numpy 2.x —from numpy import InfImportError when the venv uses--system-site-packages; (b) GDN fusion concatenating single-elementweight_scale_2/input_scalebuffers into a 4-element vector, which TRT's ONNX parser rejects:importTRT_FP4DynamicQuantize: Scale input must be a scalar— fixed by keeping numel==1 buffers unconcatenated infuse_gdn_input_projections).llm_build --onnxDir OUT/llm --engineDir ENG --maxBatchSize 1 --maxInputLen 4096 --maxKVCacheCapacity 32768 --maxVerifyTreeSize 4 --specBase→
[ERROR] IBuilder::buildSerializedNetwork: Error Code 1: Internal Error
(Not implemented for node type PLUGIN_V3. In removeEmptyProducersFromSubgraph
at optimizer/myelin/rewrite/removeEmptyTensors.cpp:257) // TRT 10.13
... removeEmptyTensors.cpp:306 // TRT 10.14
- The MTP draft engine (
--specDrafton OUT/mtp_draft) builds fine on both TRT versions. The draft ONNX contains onlytrt_edgellm::AttentionPlugin x1; the base ONNX additionally hastrt_edgellm::gated_delta_net x48andtrt_edgellm::causal_conv1d x48, so the GDN plugins appear to be the PLUGIN_V3 nodes Myelin cannot handle. - The experimental direct (ONNX-less) builder fails identically at the same Myelin pass, so this is not ONNX-path specific.
Questions
- Which TRT version was Qwen3.8-27B Day-0 support validated against on Thor? Does it require JetPack 7.2 (CUDA 13.2 / TRT 10.16)?
- Is there a builder flag/env to route GDN plugin nodes around the failing Myelin rewrite pass?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with tensorrt-edgellm-export and llm_build using the reported Qwen3.8-27B checkpoint and TensorRT versions. Compare the base and MTP draft ONNX graphs, especially the gated_delta_net and causal_conv1d PLUGIN_V3 nodes, then determine whether a supported TensorRT version or builder workaround avoids the Myelin rewrite failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, embedded-iot
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100