CopyPackedKernel is taking too long, and how to optimize it
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
I have a model that uses a slice operator for feature crossing, but it turns out that the slice operator calls the CopyPackedKernel API, and it consumes a lot of time. I also re-implemented the slice operator myself, but the same result was achieved,I don't know when CopyPackedKernel is running, how to optimize it.
nsys profile -o test --stats=true python infer.py -e test.plan
output:
Time(%) Total Time Instances Average Minimum Maximum Name
------- -------------- ---------- -------------- -------------- -------------- --------------------------------------------------------------------------------------------------------------------
99.9 9863089 2835 3479.0 3423 3872 void genericReformat::copyPackedKernel<float, float, true, true, genericReformat::IdentityCoordMapper<4>, 4>(unsigned int, unsigned int, void const*, genericReformat::ArrayN<4>, genericReformat::ArrayNWithReducedDivisors<4>, genericReformat::ArrayN<4>, int, int, int, float const*, void*, genericReformat::ArrayN<4>, genericReformat:
0.1 7264 3 2421.3 2304 2656 slice(float const*, float*, int, int, int, int)
Environment
TensorRT Version: 7.2.2.1
NVIDIA GPU: T4
NVIDIA Driver Version: 450.51.06
CUDA Version: 11.1
CUDNN Version:
Operating System:
Python Version (if applicable): 3.8
Tensorflow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if so, version): Container 20.12
I need help, thank you very much.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reported nsys command, infer.py, and test.plan, then trace where the slice operator invokes CopyPackedKernel in the TensorRT 7.2.2.1 environment. Compare the kernel's profiling cost with the slice call and determine what optimization is appropriate; the issue does not define a specific success criterion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100