intel / intel/xFasterTransformer
1st token latency has poor performance than other framework
Open
- Dominant language
- C++
- Stars
- 435
- Forks
- 76
- PR merge metrics
- No merged PRs in 30d
Description
when i test the model "DeepSeek-R1-Distill-Qwen-7B", the TTFT metrix worse than openvino，I don't know if it's normal. If so, is there any way to improve this performance
Environment：
CPU：2x8592+
Memory: 16x Hynix HMCG94AGBRA179N 64G DDR5 2Rx4
Benchmark command:
Contributor guide
Assessment
This issue has not been assessed yet.