intel / intel/xFasterTransformer
[bug] Segmentation fault occurs at large batch sizes
- Dominant language
- C++
- Stars
- 435
- Forks
- 76
- PR merge metrics
- No merged PRs in 30d
Description
Segmentation fault occurs at large batch sizes
1. _Command Line:_
./run_benchmark.sh -m llama-7b -d bf16 -s 1 -bs 100 -in 512 -out 256 -i 1
_Functions with errors:_
onednn_amx_sgemm_f32bf16f32_compute_biasadd
_Matmul matrix shape:_
M = 51200, N = 12288, K= 4096, transA = 0,alpha=1.000000, lda=4096, beta=0.000000,ldc=12288
_oneDNN_verbose:_
onednn_verbose,info,oneDNN v3.2.0 (commit 04b180b9a58a78cf1a1cd2329671a5060c2be8de)
onednn_verbose,info,cpu,runtime:OpenMP,nthr:48
onednn_verbose,info,cpu,isa:Intel AVX-512 with float16, Intel DL Boost and bfloat16 support and Intel AMX with bfloat16 and 8-bit integer support
onednn_verbose,info,gpu,runtime:none
onednn_verbose,info,prim_template:operation,engine,primitive,implementation,prop_kind,memory_descriptors,attributes,auxiliary,problem_desc,exec_time
2. _Command Line:_
./run_benchmark.sh -m llama-7b -d bf16 -s 1 -bs 100 -in 32 -out 32 -i 1
_Functions with errors:_
hpj::Matrix &input, hpj::Matrix &output, hpj::Matrix &residential, bool isMaster) {
TimeLine t("DownProj")
**assert(input.Rows() == output.Rows()); (ASSERT FAILED input.Cols()=22016, downWeight.Rows()=11008;)**
_Matmul matrix shape:_
M = 3200, N = 12288, K= 4096, transA = 0,alpha=1.000000, lda=4096, beta=0.000000,ldc=12288
_Verbose:_
xft_verbose,exec,cpu,api,onednn_amx_sgemm_f32bf16f32_compute_biasadd,m3200n12288k4096,29.308059
xft_verbose,exec,cpu,api,onednn_amx_sgemm_f32bf16f32_compute_residential,m3200n4096k4096,12.953664
xft_verbose,exec,cpu,api,onednn_amx_sgemm_f32bf16f32_compute,m3200n22016k4096,42.813326
Contributor guide
Assessment
This issue has not been assessed yet.