microsoft / microsoft/TransformerCompression
Issues with LLAMA 3-8B-Instruct model
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 460
- Forks
- 58
- Avg merge
- 8h 40m
- Merged PRs (30d)
- 1
Description
Hi
I get this error while using SliceGPT for the LLama Instruct models:
ValueError: rope_scaling must be a dictionary with two fields, type and factor, got {'factor': 8.0, 'low_freq_factor': 1.0, 'high_freq_factor': 4.0, 'original_max_position_embeddings': 8192, 'rope_type': 'llama3'}
To overcome this, I updated the Transformers package version to 4.44.2. However, slice GPT is not compatible with this transformer version. Can anyone please help in solving this issue?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported rope_scaling error with the LLAMA 3-8B-Instruct model and inspect SliceGPT's Transformers compatibility. Done means the model can be processed without this error on a supported Transformers version.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100