NVIDIA / NVIDIA/TensorRT

!config.getFlag(BuilderFlag::kFP16). ) failure of TensorRT 10.6 when running demo_txt2img_flux.py on GPU h20

Open
#4,253 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Module:Demo triaged
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

I want to run this demo in torch.float16, so I change utilities.py:
engine = engine_from_network(
network,
config=CreateConfig(fp16=True,
bf16=False,
tf32=False,
int8=False,
fp8=False,
refittable=enable_refit,
profiles=[p],
load_timing_cache=timing_cache,
**extra_build_args
),
save_timing_cache=timing_cache
)

error:
[E] IBuilder::buildSerializedNetwork: Error Code 3: API Usage Error (Parameter check failed, condition: !config.getFlag(BuilderFlag::kFP16). )
[!] Invalid Engine. Please ensure the engine was built correctly
polygraphy.exception.exception.PolygraphyException: Invalid Engine. Please ensure the engine was built correctly
/usr/lib/python3.10/tempfile.py:1008: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpin7obfa_'>

Environment

cuda-python 12.6.0
cupy-cuda12x 13.2.0
dask-cuda 24.8.0
nvidia-dali-cuda120 1.41.0
pytorch-triton 3.0.0+dedb7bdf3
torch 2.5.0a0+b465a5843b.nv24.9
torch_tensorrt 2.5.0a0
torchprofile 0.0.4
torchvision 0.20.0a0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with demo_txt2img_flux.py and the engine_from_network configuration in utilities.py, then reproduce the TensorRT 10.6 build on the stated H20 and software environment. Trace why enabling torch.float16 produces the kFP16 API Usage Error; done means the demo builds a valid engine and runs successfully with the requested precision.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.