intel/xFasterTransformer

View on GitHub

Stars: 435
Forks: 76
Open beginner issues: 0
Indexed issues: 20

Dominant language: C++
License: Apache-2.0
Last GitHub push: Aug 20, 2025
Latest indexed: Sep 17, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Open All

20 open issues indexed

run llama2 in two CPU meet error when set dtype int8 and bf16_int8 Open

bug

intel/xFasterTransformer#56 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
run webdemo meet error when set dytpe bf16_int8 and bf16_fp16 Open

bug

intel/xFasterTransformer#59 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[bug] Segmentation fault occurs at large batch sizes Open

intel/xFasterTransformer#140 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
chatglm3 6b error Open

intel/xFasterTransformer#335 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
performance issue for opt-1.3b with BS=1 BF16 Open

intel/xFasterTransformer#339 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[output issue] found mistakes in llama-3-70b output by bf16_int4 during benchmark Open

intel/xFasterTransformer#413 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Can we summarize the meanings of data type like bf16_fp16?, for example, what's activation data type and output data type, what's the computing instruction? Open

documentation

intel/xFasterTransformer#414 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
oneDNN and XFT performance Open

intel/xFasterTransformer#471 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Qwen2.5-0.5B-Instruct quantization with gptq error Open

intel/xFasterTransformer#480 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
qwen2.5 run error Open

intel/xFasterTransformer#482 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Older gcc support Open

intel/xFasterTransformer#486 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Feature] Support Medusa decode (Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads) Open

intel/xFasterTransformer#489 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Got a wrong result with DeepSeek-Distill-Qwen-7b while running vllm serving with OMP_NUM_THREADS=16 Open

intel/xFasterTransformer#491 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
1st token latency has poor performance than other framework Open

intel/xFasterTransformer#509 · 12 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
how benchmark compute the multi process proformance? Open

intel/xFasterTransformer#510 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
why benchmark choose create multi process to test performance metrix? Open

intel/xFasterTransformer#511 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
undefined reference to Qwen2LLM Open

intel/xFasterTransformer#514 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Which instruction set will the CPU use? Open

intel/xFasterTransformer#515 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
build_async_engine_client_from_engine_args cannot import Open

intel/xFasterTransformer#516 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
我这个CPU可以跑大模型吗？ Open

intel/xFasterTransformer#521 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026

intel/xFasterTransformer

Get new issues in your inbox