Issue with Gemma 7b
- Dominant language
- TypeScript
- Stars
- 979
- Forks
- 132
- Avg merge
- 9h 23m
- Merged PRs (30d)
- 8
Description
## Describe the bug
I tried loading Gemma 7b, and finally got the following long string of random words instead of a response after the model was loaded.
## To Reproduce
Steps to reproduce the behavior:
start AI Playground
Go to Answer tab
Select Gemma 7b model from the list of models,
type in a query, (hello,)
get the following long string of nonsense
## Expected behavior
I expected to get a response from the LLM
## Screenshots
[ai-backend]: 2024-11-16 18:56:46,907 - INFO - WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on http://127.0.0.1:59000
2024-11-16 18:56:46,907 - INFO - Press CTRL+C to quit
[ai-backend]: 2024-11-16 18:56:47,162 - INFO - 127.0.0.1 - - [16/Nov/2024 18:56:47] "POST /api/init HTTP/1.1" 200 -
[ai-backend]: 2024-11-16 18:56:47,167 - INFO - 127.0.0.1 - - [16/Nov/2024 18:56:47] "POST /api/getGraphics HTTP/1.1" 200 -
[ai-backend]: 2024-11-16 18:57:03,637 - INFO - 127.0.0.1 - - [16/Nov/2024 18:57:03] "POST /api/checkModelExist HTTP/1.1" 200 -
[ai-backend]: 2024-11-16 18:57:03,757 - INFO - 127.0.0.1 - - [16/Nov/2024 18:57:03] "POST /api/llm/chat HTTP/1.1" 200 -
[ai-backend]: Gemma's activation function should be approximate GeLU and not exact GeLU.
Changing the activation function to `gelu_pytorch_tanh`.if you want to use the legacy `gelu`, edit the `model.config` to set `hidden_activation=gelu` instead of `hidden_act`. See https://github.com/huggingface/transformers/pull/29402 for more details.
Loading checkpoint shards: 0%| | 0/4 [00:00
Contributor guide
Assessment
This issue has not been assessed yet.