henrywoo/pyllama

View on GitHub

LLaMA: Open and Efficient Foundation Language Models

Stars: 2.8k
Forks: 296
Open beginner issues: 0
Indexed issues: 61

Dominant language: Python
License: GPL-3.0
Last GitHub push: Apr 25, 2023
Latest indexed: Sep 17, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: good first issue
PR merge metrics: No merged PRs in 30d

Open All

61 open issues indexed

is py-itree available for Windows? Open

henrywoo/pyllama#8 · 14 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Running web_server.py on Multi GPU instance. Open

henrywoo/pyllama#9 · 7 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Sorry,I can't run Open

henrywoo/pyllama#12 · 8 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
M1 inference Open

henrywoo/pyllama#24 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Error when download models Open

henrywoo/pyllama#26 · 5 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
How to run llama_quant without downloading models from huggingface ? Open

enhancement good first issue

henrywoo/pyllama#27 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Model does not split for 65B Open

henrywoo/pyllama#28 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Model mismatch for 13B Open

henrywoo/pyllama#31 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
ModuleNotFoundError: No module named 'quant_cuda' Open

henrywoo/pyllama#37 · 14 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
"torch.cuda.OutOfMemoryError: CUDA out of memory" when I'm *not* out of memory Open

henrywoo/pyllama#38 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Error trying Quantize 7B model to 2-bit Open

henrywoo/pyllama#41 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
AttributeError: module 'numpy' has no attribute 'array' Open

henrywoo/pyllama#43 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
Share your evaluate result Open

henrywoo/pyllama#45 · 3 comments · 7 reactions · 0 assignees · Indexed Sep 17, 2026
How can I input prompt when I use multi GPU? Open

henrywoo/pyllama#46 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
pyllama/downloads returns empty folders Open

question

henrywoo/pyllama#47 · 34 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Error Downloading Models from Community on Winodws Open

bug

henrywoo/pyllama#50 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Meaningless Prediction in 13B 2bit Open

henrywoo/pyllama#52 · 3 comments · 3 reactions · 0 assignees · Indexed Sep 17, 2026
Docker install Open

henrywoo/pyllama#53 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Downloading get stuck in infinite loop Open

henrywoo/pyllama#56 · 13 comments · 5 reactions · 0 assignees · Indexed Sep 17, 2026
already quantize to 4bit and get the model pyllama-7B4b.pt，but can not run in RTX3080. report torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 86.00 MiB (GPU 0; 10.00 GiB total capacity; 9.24 GiB already allocated; Open

henrywoo/pyllama#57 · 2 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
Quantization with "groupsize" makes the results completely wrong. Open

henrywoo/pyllama#58 · 8 comments · 2 reactions · 0 assignees · Indexed Sep 17, 2026
Quantize Original LLaMA Model Files Open

henrywoo/pyllama#60 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Any way to infer a quantized model on multi GPUs? Open

henrywoo/pyllama#61 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 17, 2026
Killed Open

henrywoo/pyllama#62 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
How to run the gradio with 30B model? and what devices are needed? please Open

henrywoo/pyllama#64 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Has black formatting been considered? Open

henrywoo/pyllama#65 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Can't see progress bar Open

henrywoo/pyllama#66 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 17, 2026
ModuleNotFoundError: No module named 'transformers' Open

henrywoo/pyllama#67 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Document if it works with CPU / Macos Open

henrywoo/pyllama#69 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Readme Should Have Inference Command to use for Quantization in Text Open

henrywoo/pyllama#72 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
quantify llama 7B, the md5 value and the model size does not equals to the value in README Open

henrywoo/pyllama#73 · 6 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
a questuon about the single GPU Inference Open

henrywoo/pyllama#74 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Can't Load Quantized Model with GPTQ-for-LLaMa Open

henrywoo/pyllama#75 · 2 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
Is there a way to skip evaluating after quantizing because it takes forever? Open

henrywoo/pyllama#77 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Quantized version link suspect Open

henrywoo/pyllama#78 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Inference Error :UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe7 in position 18: invalid continuation byte" Open

henrywoo/pyllama#80 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Cannot run on Mac with Python 3.11.3 Open

henrywoo/pyllama#81 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Strange characters Open

henrywoo/pyllama#82 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
How to run 13B model in a single GPU just by inference.by? Open

henrywoo/pyllama#84 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Apply Delta failed Open

henrywoo/pyllama#85 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 17, 2026
Run 'inference.py' and 'model parallel group is not initialized' Open

henrywoo/pyllama#86 · 7 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
aria2c 'magnet:?xt=urn:btih:ZXXDAUWYLRUXXBHUYEMS6Q5CE5WA3LVA&dn=LLaMA' not working Open

henrywoo/pyllama#87 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
RecursionError running llama.download Open

henrywoo/pyllama#90 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Quantize issue Open

henrywoo/pyllama#92 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Does this include the GPTQ quantization tricks? Open

henrywoo/pyllama#94 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Randomly get shape mismatch error Open

henrywoo/pyllama#95 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Try Modular - Mojo Open

henrywoo/pyllama#96 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
gptq github Open

henrywoo/pyllama#97 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
parameter inncorrect when I run make command Open

henrywoo/pyllama#98 · 0 comments · 2 reactions · 0 assignees · Indexed Sep 17, 2026
an operation was attempted on something that is not a socket Open

henrywoo/pyllama#99 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026

henrywoo/pyllama

Get new issues in your inbox