[Feature Request] Need Matmul Attention layer instead of Einsum to support GPU running
Open
- Dominant language
- Python
- Stars
- 200
- Forks
- 45
- PR merge metrics
- No merged PRs in 30d
Description
Einsum kernel couldn't' be lowered to cudnn GEMM. The computing performance is seriously affected.
Can you believe it? JAX(Flax or Praxis) attention layers are even slower than Tensorflow version (not Keras)!
Contributor guide
Assessment
This issue has not been assessed yet.