pytorch / pytorch/TensorRT

❓ [Question] How to decide if an Op should support dynamic shape or not

Open
#3,224 15 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question story: Documentation & Examples
Dominant language
Python
Stars
3k
Forks
410
Avg merge
3d 18h
Merged PRs (30d)
78

Description

❓ Question

Since only part of the ops support dynamic shapes, and some are not. What's the criteria to decide if an op supports dynamic shape or not?

For some existing ops, which are not marked as supports_dynamic_shapes=True, can I write a converter that wraps the existing converter, and mark my own converter with high priority? Is this the recommended way?

or should I just turn on assume_dynamic_shape_support, which seems to be a flag globally for all converters ?

What you have already tried

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

  • PyTorch Version (e.g., 1.0): 2.4.1
  • CPU Architecture: x86_64
  • OS (e.g., Linux): linux
  • How you installed PyTorch (conda, pip, libtorch, source): pip
  • Build command you used (if compiling from source):
  • Are you using local sources or building from archives:
  • Python version: 3.11.9
  • CUDA version: 12.1
  • GPU models and configuration: Nvidia L4
  • Any other relevant information:

Additional context

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the converter implementations and the handling of supports_dynamic_shapes and assume_dynamic_shape_support. Trace how converter priority and dynamic-shape checks are applied, then document the criteria for support and when a wrapper converter or the global flag is appropriate.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
compilers, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.