Kernel dies when using llama-2-13b
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12.5k
- Forks
- 991
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 10
Description
I'm trying to run a python notebook with this version of llama: https://huggingface.co/TheBloke/Llama-2-13B-chat-GGUF/resolve/main/llama-2-13b-chat.Q5_0.gguf
but I keep running into the issue "The Kernel crashed while executing code in the the current cell or a previous cell." When I try running a prompt on the CLI, I receive no output. Both of these examples work fine using https://huggingface.co/TheBloke/Llama-2-7b-Chat-GGUF/resolve/main/llama-2-7b-chat.Q8_0.gguf
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the crash with the linked Llama-2-13B GGUF model in the Python notebook and on the CLI, then compare both results with the working 7B model. Done means the 13B model produces output without crashing the kernel in both usage paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, python
- Domain
- ai, cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100