NVIDIA / NVIDIA/TensorRT-Edge-LLM
Version mismatch while running engine files
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 563
- Forks
- 135
- Avg merge
- 14h 13m
- Merged PRs (30d)
- 1
Description
Hi
I am using tensorrt-edgellm for quantising the Cosmos reason 2 model into fp8.
Once done with quantization, I am running in a different docker and getting this error -
[08/25/2026-14:46:02] [TRT] [E] IRuntime::deserializeCudaEngine: Error Code 1: Serialization (Serialization assertion stdVersionRead == kSERI
ALIZATION VERSION failed.Version tag does not match. Note: Current Version: 243, Serialized Engine Version: 240 In stdArchiveReaderInitCommon
at /src/runtime/deserialization/stdArchiveReader.cpp:49)
,2026-08-25 14:46:02,935][ main_ _][ERROR] - [rank: 0] Hybrid evaluation failed: 'NoneType' object has no attribute "create execution context
I thought this is the tensorrt version issue so I used the same tensorrt==11.2.1.2 but still gor the issue.
Here are the key points -
- Engine for cosmos reason 2 is built using tensorrt-edgellm
- I am trying to run in a different docker with no tensorrt-edgellm but only tensorrt
- Both the docker has same tensorrt version.
Question -
- Is it possible to build engine using tensorrtedgellm and run it separately using just tensorrt on the same platform?
- What does the error says exactly? Which version of tensorrt corresponds to engine version 240 amd 243.
Thanks
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing the TensorRT-Edge-LLM quantization Docker environment with the separate TensorRT runtime environment described in the issue, focusing on the serialized engine versions 240 and 243. No source file or test is named; done would require determining whether the engine is compatible with standalone TensorRT and documenting the version or runtime requirements.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- ai-infra-agents, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100