abertsch72 / abertsch72/unlimiformer
Hardware Requirement for Running Llama-2 inferences
Open
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 78
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I successfully ran the inferences with Llama-2-7b and unlimiformer but ran into memory errors when jumped to larger models. What are the minimum GPU memory requirements for running 13b and 70b models? Thank you!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.