Why `model_max_length` from vicuna 1.1 and vicuna 1.3 are different?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
I tested vicuna 1.1 on text-generation-web-ui and the model can handle long input even bigger than `max_position_embeddings`. Now, in vicuna 1.3, I kept getting warning when my input is longer than 2048 tokens. When I check the `tokenizer_config.json`, vicuna 1.1 has a very large value for `model_max_length` which is `1000000000000000019884624838656` rather than vicuna 1.3 which the value is `2048`.
Why is this happening? Is this intentional? What could be the problem if I somehow change the value manually?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing the Vicuna 1.1 and 1.3 tokenizer_config.json values with their max_position_embeddings settings, then trace how text-generation-web-ui uses model_max_length for input warnings. Done means documenting whether the difference is intentional and the consequences of manually changing the value.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100