NVIDIA / NVIDIA/TensorRT-Edge-LLM

Version mismatch while running engine files

Open
#194 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
563
Forks
135
Avg merge
14h 13m
Merged PRs (30d)
1

Description

Hi

I am using tensorrt-edgellm for quantising the Cosmos reason 2 model into fp8.

Once done with quantization, I am running in a different docker and getting this error -


[08/25/2026-14:46:02] [TRT] [E] IRuntime::deserializeCudaEngine: Error Code 1: Serialization (Serialization assertion stdVersionRead == kSERI
ALIZATION VERSION failed.Version tag does not match. Note: Current Version: 243, Serialized Engine Version: 240 In stdArchiveReaderInitCommon
at /src/runtime/deserialization/stdArchiveReader.cpp:49)
,2026-08-25 14:46:02,935][
main_ _][ERROR] - [rank: 0] Hybrid evaluation failed: 'NoneType' object has no attribute "create execution context


I thought this is the tensorrt version issue so I used the same tensorrt==11.2.1.2 but still gor the issue.

Here are the key points -

  1. Engine for cosmos reason 2 is built using tensorrt-edgellm
  2. I am trying to run in a different docker with no tensorrt-edgellm but only tensorrt
  3. Both the docker has same tensorrt version.

Question -

  1. Is it possible to build engine using tensorrtedgellm and run it separately using just tensorrt on the same platform?
  2. What does the error says exactly? Which version of tensorrt corresponds to engine version 240 amd 243.

Thanks

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the TensorRT-Edge-LLM quantization Docker environment with the separate TensorRT runtime environment described in the issue, focusing on the serialized engine versions 240 and 243. No source file or test is named; done would require determining whether the engine is compatible with standalone TensorRT and documenting the version or runtime requirements.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
ai-infra-agents, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.