optimization profile is missing values for shape input
Open
Nobody has claimed this yet.
triaged
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
I tried to convert the onnx model using the following command
shapes = {
'position' : [(1, 1), (1, 1000), (1, 2000)],
'inputs_embeds': [(1, 1, 1792), (1, 1000, 1792), (1, 2000, 1792)],
'past_key_in0': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_value_in0': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_key_in1': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_value_in1': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_key_in2' : [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_value_in2': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_key_in3': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_value_in3': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_key_in4': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_value_in4': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_key_in5': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_value_in5': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_key_in6': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_value_in6': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_key_in7': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_value_in7': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_key_in8': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_value_in8': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_key_in9': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_value_in9': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_key_in10': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_value_in10': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_key_in11': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'past_value_in11': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
'kv_slice': [(1,), (1,), (1,)],
}
for order, (name, shape_list) in enumerate(shapes.items()):
print(order, name, shape_list)
min_shape, opt_shape, max_shape = shape_list
profile.set_shape(name, min_shape, opt_shape, max_shape)
config.add_optimization_profile(profile)
engine = builder.build_serialized_network(network, config)
However, I got the error [TRT] [E] 4: [network.cpp::validate::3584] Error Code 4: Internal Error (kv_slice: optimization profile is missing values for shape input)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided Python reproduction and the TensorRT network validation error for the kv_slice shape input. Trace how the optimization profile handles the listed inputs and how kv_slice is declared in the network. Done means the same profile builds successfully without the missing-values error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100