intel / intel/xFasterTransformer
performance issue for opt-1.3b with BS=1 BF16
Open
- Dominant language
- C++
- Stars
- 435
- Forks
- 76
- PR merge metrics
- No merged PRs in 30d
Description
test opt 1.3 model on EMR platform with 52c. the performance is not right with BS=1. the gap between BS=1 and BS =2 is too big.
`numactl -C 0-51 -m 0 ./run_benchmark.sh -m opt-1_3b -d bf16 -s 1 -bs 1 -in 128 -out 15 -i 10`
the results
BS=1
BS=2
Contributor guide
Assessment
This issue has not been assessed yet.