Lightning-AI / Lightning-AI/lit-llama
why cannot the generate function be used twice
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.1k
- Forks
- 517
- PR merge metrics
- No merged PRs in 30d
Description
When i invoke the generate function twice with different idx,it appears the fault described as "RuntimeError: The expanded size of the tensor (181) must match the existing size (168) at non-singleton dimension 3. Target sizes: [1, 32, 68, 181]. Tensor sizes: [1, 1, 68, 168]"
should i reset the model' arguements or reload the model?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the generate function and reproduce two calls with different idx values, recording the tensor shapes from the reported error. Check whether the second call succeeds without reloading or resetting the model; done means the repeated-call behavior is fixed or the required reset or reload behavior is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100