Hardware Requirement for Running Llama-2 inferences

Hi, I successfully ran the inferences with Llama-2-7b and unlimiformer but ran into memory errors when jumped to larger models. What are the minimum GPU memory requirements for running 13b and 70b models? Thank you!

Look at the project's README or documentation for any existing hardware notes. Check the model loading code and inference scripts to see how GPU memory is managed. The answer likely involves calculating memory based on model parameters and context length, so understanding the library's memory usage patterns is key.

Escrita pelo modelo de indexação a partir do texto da issue.