Generation speed issue

I load llama2 model like example successfully but the speed to generate text is really slow.


[1] I'm not sure it use mps to accelerate generation.
How to confirm it?
[2] Is there a smaller LLM than 7B?

Here is my env
- Macbook Air / M2 / 16GB / Sonoma 14.5
- Xcode 15.4
- ckpt: coreml-projects/Llama-2-7b-chat-coreml

Reproduce the slow generation on the stated MacBook Air M2 environment using the llama2 example and the listed coreml-projects/Llama-2-7b-chat-coreml checkpoint. Determine whether MPS acceleration is active and document the observed generation speed; the issue does not mention any source files or tests, and completion criteria are not defined beyond answering the two questions.

Written by the indexing model from the issue text.