Lightning-AI/lit-llama

View on GitHub

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

Stars: 6.1k
Forks: 517
Open beginner issues: 0
Indexed issues: 100

Dominant language: Python
License: Apache-2.0
Last GitHub push: Jul 1, 2025
Latest indexed: Sep 19, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: good first issue help wanted
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

100 open issues indexed

Broken link redirection Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

Lightning-AI/lit-llama#493 · 1 comment · Opened Feb 5, 2025
fine-tuning time compute Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

Lightning-AI/lit-llama#492 · Opened Oct 15, 2024
add special token Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

Lightning-AI/lit-llama#490 · Opened Jul 2, 2024
Where is tokenizer.model? tokenizer path Open

Difficulty 1/5 Under an hour Newbie friendliness 25/100

Lightning-AI/lit-llama#488 · 1 comment · Opened Apr 24, 2024
Using llama3 through lit lama Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

Lightning-AI/lit-llama#487 · 5 comments · Opened Apr 22, 2024
How to convert lit-llama pretrained model to HF format? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

Lightning-AI/lit-llama#486 · 1 comment · Opened Apr 16, 2024
why cannot the generate function be used twice Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Lightning-AI/lit-llama#485 · Opened Apr 2, 2024
Converting from lit-llama to HF checkpoint? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

Lightning-AI/lit-llama#484 · 2 reactions · Opened Mar 24, 2024
it seems that hash of traindata is lost, so it's impossible to continue finetune after stop Open

Difficulty 3/5 1-2 days Newbie friendliness 42/100

Lightning-AI/lit-llama#483 · Opened Mar 16, 2024
`PackedDatasetBuilder` does not separate with `sep_token` Open

Difficulty 3/5 1-2 days Newbie friendliness 38/100

Lightning-AI/lit-llama#482 · Opened Mar 6, 2024
Beam search generation Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

Lightning-AI/lit-llama#477 · Opened Dec 27, 2023
Error: git submodule update --init --recursive -q did not run successfully Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

Lightning-AI/lit-llama#475 · 1 comment · Opened Dec 14, 2023
Ban some tokens Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

Lightning-AI/lit-llama#474 · Opened Nov 22, 2023
Can I use Lightning fabirc to pre train llama2 on v100? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

Lightning-AI/lit-llama#473 · 1 comment · Opened Nov 17, 2023
How to quantize LLama in fine-tuning ? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

Lightning-AI/lit-llama#470 · Opened Nov 13, 2023
How to convert hf weight of 70b to lit-lamma weights? Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Lightning-AI/lit-llama#469 · Opened Nov 10, 2023
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0 Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Lightning-AI/lit-llama#467 · Opened Nov 2, 2023
Why is LLaMA response to queries in the conversation so wrong? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

Lightning-AI/lit-llama#466 · Opened Oct 31, 2023
[question] nan loss value and run time error Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

Lightning-AI/lit-llama#463 · Opened Oct 28, 2023
[question] assert lora_path.is_file() error Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

Lightning-AI/lit-llama#462 · 1 comment · Opened Oct 23, 2023
Question about 'validating...' from lora.py Open

Difficulty 1/5 Under an hour Newbie friendliness 25/100

Lightning-AI/lit-llama#461 · Opened Oct 21, 2023
[question] error message while finetuning Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

Lightning-AI/lit-llama#460 · 2 comments · Opened Oct 21, 2023
Adapter finetuning do not run on two cards (A100 40G) Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

Lightning-AI/lit-llama#459 · Opened Oct 2, 2023
Mistral Model Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

Lightning-AI/lit-llama#458 · 1 comment · Opened Oct 1, 2023
When I finetuned the model, an error occurred during the decoding process: IndexError: Out of range: piece id is out of range. Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

Lightning-AI/lit-llama#457 · Opened Sep 30, 2023
Looking for LLaMA 2? Open

Difficulty 1/5 Under an hour Newbie friendliness 25/100

Lightning-AI/lit-llama#452 · 3 comments · Opened Aug 25, 2023
Only add a linear layer to LLaMA without any computation degrade the performance Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

Lightning-AI/lit-llama#451 · Opened Aug 17, 2023
(documentation) How do I know if generate.py is running on GPU / GPU configuration Open

Difficulty 2/5 1-3 hours Newbie friendliness 25/100

Lightning-AI/lit-llama#449 · 1 comment · Opened Aug 15, 2023
multi gpus for full finetune Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

Lightning-AI/lit-llama#446 · 3 comments · Opened Aug 10, 2023
TypeError: super(type, obj): obj must be an instance or subtype of type Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

Lightning-AI/lit-llama#445 · 3 comments · Opened Aug 9, 2023
No response after training an epoch Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

Lightning-AI/lit-llama#443 · 2 comments · Opened Aug 8, 2023
converting Adapter to huggingface format Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

Lightning-AI/lit-llama#440 · Opened Aug 7, 2023
This codebase has so many errors it is completely useless and unusable Open

Difficulty 4/5 3-5 days Newbie friendliness 15/100

Lightning-AI/lit-llama#438 · 2 comments · Opened Aug 7, 2023
reset_cache() Decrease the Generation Quality of Consecutive Inferences Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

Lightning-AI/lit-llama#437 · Opened Aug 7, 2023
combine adapter weights with the base model Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

Lightning-AI/lit-llama#433 · Opened Jul 29, 2023
Question about FlashAttention and KV-cache Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

Lightning-AI/lit-llama#430 · Opened Jul 23, 2023
Why didn't use matrix multiplication in the implememtation of LoRA? Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

Lightning-AI/lit-llama#429 · 2 reactions · Opened Jul 20, 2023
ImportError when trying to use 'Linear8bitLt' from 'lit_llama.quantization' Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

Lightning-AI/lit-llama#421 · 2 comments · Opened Jul 11, 2023
Training on GPT-4-LLM Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

Lightning-AI/lit-llama#420 · Opened Jul 11, 2023
Issue with "kv_cache" while using modified generate/lora.py for a list of inputs Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

Lightning-AI/lit-llama#418 · 5 comments · Opened Jul 7, 2023
How to train 13B version on 8bit with LoRA Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

Lightning-AI/lit-llama#413 · 2 comments · Opened Jun 30, 2023
Resume takes much more GPU RAM? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

Lightning-AI/lit-llama#411 · Opened Jun 29, 2023
How to migrate the finetune/full.py to a multi-node and multi-gpu environment? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

Lightning-AI/lit-llama#409 · Opened Jun 27, 2023
【solved】use adpater_v2.py fine-tuning llama 13B error Open

Difficulty 4/5 3-5 days Newbie friendliness 15/100

Lightning-AI/lit-llama#405 · 4 comments · Opened Jun 21, 2023
getting random tokens after finetuning Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

Lightning-AI/lit-llama#403 · Opened Jun 20, 2023
Error while running modified prepare_alpaca.py on Linux Mint 21.1 Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Lightning-AI/lit-llama#401 · 6 comments · Opened Jun 20, 2023
OOM when fully finetuning llama-7B using the deepspeed strategy Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

Lightning-AI/lit-llama#397 · Opened Jun 19, 2023
Group data by length to reduce wasted computation Open

Difficulty 3/5 1-2 days Newbie friendliness 38/100

Lightning-AI/lit-llama#396 · 3 comments · 2 reactions · Opened Jun 18, 2023
Is it possible to further reduce the RAM? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

Lightning-AI/lit-llama#395 · 5 comments · Opened Jun 17, 2023
Questions about finetuning Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

Lightning-AI/lit-llama#394 · Opened Jun 16, 2023

Lightning-AI/lit-llama

Get new issues in your inbox