intel / intel/AI-Playground

Issue with Gemma 7b

Open
#102 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
TypeScript
Stars
979
Forks
132
Avg merge
9h 23m
Merged PRs (30d)
8

Description

## Describe the bug

I tried loading Gemma 7b, and finally got the following long string of random words instead of a response after the model was loaded.

## To Reproduce

Steps to reproduce the behavior:
start AI Playground
Go to Answer tab
Select Gemma 7b model from the list of models,
type in a query, (hello,)
get the following long string of nonsense

## Expected behavior

I expected to get a response from the LLM

## Screenshots

[ai-backend]: 2024-11-16 18:56:46,907 - INFO - WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on http://127.0.0.1:59000
2024-11-16 18:56:46,907 - INFO - Press CTRL+C to quit

[ai-backend]: 2024-11-16 18:56:47,162 - INFO - 127.0.0.1 - - [16/Nov/2024 18:56:47] "POST /api/init HTTP/1.1" 200 -

[ai-backend]: 2024-11-16 18:56:47,167 - INFO - 127.0.0.1 - - [16/Nov/2024 18:56:47] "POST /api/getGraphics HTTP/1.1" 200 -

[ai-backend]: 2024-11-16 18:57:03,637 - INFO - 127.0.0.1 - - [16/Nov/2024 18:57:03] "POST /api/checkModelExist HTTP/1.1" 200 -

[ai-backend]: 2024-11-16 18:57:03,757 - INFO - 127.0.0.1 - - [16/Nov/2024 18:57:03] "POST /api/llm/chat HTTP/1.1" 200 -

[ai-backend]: Gemma's activation function should be approximate GeLU and not exact GeLU.
Changing the activation function to `gelu_pytorch_tanh`.if you want to use the legacy `gelu`, edit the `model.config` to set `hidden_activation=gelu` instead of `hidden_act`. See https://github.com/huggingface/transformers/pull/29402 for more details.

Loading checkpoint shards: 0%| | 0/4 [00:00

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.