InternLM / InternLM/lmdeploy

[bug] group_size = 64 has bug

Open
#322 5 comments 0 reactions 1 assignee Claimed by @irexyc View on GitHub
backlog
Dominant language
Python
Stars
8.1k
Forks
748
Avg merge
6d 2h
Merged PRs (30d)
54

Description

只修改这两行代码,LLaMa-2-7B 模型无法得到正确的输出。
` Generate<64, Ops>(kernels_); group_sizes_.push_back(64);`

FP16 的输出:
@Input:
The first time I saw the movie, I was like, 'Oh my God,
_Output:
this is so cool.' I was like,

INT4 GroupSize 64 的输出:
@Input:
The first time I saw the movie, I was like, 'Oh my God,
_Output:
I was a little, I was a little

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.