Lightning-AI / Lightning-AI/litgpt

chatting with mistral generates answer with no spaces

Open
#1,822 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug good first issue help wanted
Dominant language
Python
Stars
13.7k
Forks
1.5k
Avg merge
15h 37m
Merged PRs (30d)
1

Description

### Bug description

steps to reproduce the issue:
1. litgpt chat checkpoints/mistralai/Mistral-7B-Instruct-v0.3 --max_new_tokens 2048
2. enter any prompt

I get answer with no space between words, i.e.: "Hello!I'mjustacomputerprogram,soIdon'thavefeelingslikeahumandoes.ButI'mheretohelpyouwithanyquestionsortasksyoumighthave!HowcanIassistyoutoday?"

Here is the full log:
```
⚡ ~ litgpt chat checkpoints/mistralai/Mistral-7B-Instruct-v0.3 --max_new_tokens 2048
{'access_token': None,
'checkpoint_dir': PosixPath('checkpoints/mistralai/Mistral-7B-Instruct-v0.3'),
'compile': False,
'max_new_tokens': 2048,
'multiline': False,
'precision': None,
'quantize': None,
'temperature': 0.8,
'top_k': 50,
'top_p': 1.0}
Now chatting with Mistral-7B-Instruct-v0.3.
To exit, press 'Enter' on an empty prompt.

Seed set to 1234
>> Prompt: hi, how are you?
>> Reply: Hello!I'mjustacomputerprogram,soIdon'thavefeelingslikeahumandoes.ButI'mheretohelpyouwithanyquestionsortasksyoumighthave!HowcanIassistyoutoday?
Time for inference: 3.55 sec total, 12.96 tokens/sec, 46 tokens
```
### What operating system are you using?

Unknown

### LitGPT Version

```

⚡ ~ pip show litgpt | grep Version
Version: 0.5.2

```

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the mentioned `litgpt chat checkpoints/mistralai/Mistral-7B-Instruct-v0.3 --max_new_tokens 2048` command on LitGPT 0.5.2. Trace the chat output path and token decoding behavior, then verify that the generated reply preserves spaces between words.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.