fast contiguous for tensor transpose
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 333
- Forks
- 150
- Avg merge
- 4d 19h
- Merged PRs (30d)
- 54
Description
We are optimizing the bert performance now and one important aspect is the contiguous op.
Our current implementation for the transpose input shape in contiguous uses a straightforward approach, but performance is bad for now. In the bert model. the contiguous occupies about 12% of the total time, but it only to change memory layout for tensors. By looking deeps, the contiguous is mainly to do transpose of tensors. So we need a faster implementation of the contiguous for transpose.
By checking online, one thing we can refer is: https://github.com/ap-hynninen/cutt
and the corresponding paper is: https://arxiv.org/pdf/1705.01598.pdf
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the current contiguous implementation and the transpose path in AMDMIGraphX, then reproduce the BERT performance case described in the issue. Read the linked CUTT project and paper for possible approaches. Done means a faster contiguous implementation for transpose-shaped tensor inputs, with the BERT workload showing improved performance.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100