The NPU cannot accelerate normally
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## ❓softmax operations cannot be accelerated by NPU
I trained a character recognition model. The image encoding part adopted the encode of transformer. During decoding, I only used one linear layer and softmax operation. When only the linear layer was output, the model could be accelerated normally, but when softmax was added, it could not be accelerated normally.
Three methods were used for softmax operation, but none of them were feasible
F.log_softmax()
torch.softmax()
output.softmax()
Contributor guide
Research direction
Start by reproducing the character-recognition model conversion and compare NPU acceleration with only the linear layer versus each of F.log_softmax(), torch.softmax(), and output.softmax(). Inspect the conversion and acceleration behavior for the softmax operation. Done means the model's softmax path accelerates normally, or the unsupported limitation is documented with a reproducible explanation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100