intel / intel/xFasterTransformer

run llama2 in two CPU meet error when set dtype int8 and bf16_int8

Open
#56 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
C++
Stars
435
Forks
76
PR merge metrics
No merged PRs in 30d

Description

the script is in the attachment.
[llama2-7b.zip](https://github.com/intel/xFasterTransformer/files/13362390/llama2-7b.zip)

the error info is shown as below
1. int8

memory node number: 16
HBM SNC4 mode
llama2-7b.sh: 17: Bad substitution
llama2-7b.sh: 17: Bad substitution
llama2-7b.sh: 17: Bad substitution
llama2-7b.sh: 17: Bad substitution
FP16 Performance
FP16 Performance
FP16 Performance
FP16 Performance
llama2-7b.sh: 17: Bad substitution
llama2-7b.sh: 17: Bad substitution
llama2-7b.sh: 17: Bad substitution
llama2-7b.sh: 17: Bad substitution
FP16 Performance
FP16 Performance
FP16 Performance
FP16 Performance
Segmentation fault (core dumped)

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 0 PID 21023 RUNNING AT ubuntu-desktop
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 1 PID 21024 RUNNING AT ubuntu-desktop
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 2 PID 21025 RUNNING AT ubuntu-desktop
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 3 PID 21026 RUNNING AT ubuntu-desktop
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 5 PID 21028 RUNNING AT ubuntu-desktop
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 6 PID 21029 RUNNING AT ubuntu-desktop
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 7 PID 21030 RUNNING AT ubuntu-desktop
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

2. bf16_int8
memory node number: 16
HBM SNC4 mode
llama2-7b.sh: 17: Bad substitution
llama2-7b.sh: 17: Bad substitution
FP16 Performance
llama2-7b.sh: 17: Bad substitution
FP16 Performance
FP16 Performance
llama2-7b.sh: 17: Bad substitution
llama2-7b.sh: 17: Bad substitution
llama2-7b.sh: 17: Bad substitution
llama2-7b.sh: 17: Bad substitution
FP16 Performance
FP16 Performance
llama2-7b.sh: 17: Bad substitution
FP16 Performance
FP16 Performance
FP16 Performance
Segmentation fault (core dumped)

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 0 PID 21300 RUNNING AT ubuntu-desktop
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 1 PID 21301 RUNNING AT ubuntu-desktop
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 2 PID 21302 RUNNING AT ubuntu-desktop
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 3 PID 21303 RUNNING AT ubuntu-desktop
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 4 PID 21304 RUNNING AT ubuntu-desktop
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 6 PID 21306 RUNNING AT ubuntu-desktop
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 7 PID 21307 RUNNING AT ubuntu-desktop
= KILLED BY SIGNAL: 9 (Killed)
===================================================================================

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.