Text generation early stop problem with Vicuna 33B v1.3
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I have found a case where 33b stops generating its response.
To reproduce:
1. Go to https://chat.lmsys.org/
2. Select vicuna-33b
3. Say "Hi! How are you?"
The response will be "Hello! As an AI language model, I don" and it will fail on the apostrophe. This problem does not occur with vicuna-13b. I've also been able to reproduce this issue locally in text-generation-webui with different parameters.
This was the first query I tried. It's too bad, I was really excited to use vicuna-33b, but I can't rely on a model which fails (seemingly) so frequently.
Edit: The problem does not exist with the preview version of vicuna-33b
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the behavior at https://chat.lmsys.org/ with vicuna-33b using the reported prompt, then compare it with vicuna-13b and the preview vicuna-33b version. Trace the serving or generation entry point for the selected model and verify that responses no longer stop at the apostrophe without breaking the existing model variants.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100