After doing all the process to generate the project. There are no Checkpoints.

First of all congrats, this looks promising.

But, after trying 5 times to finetune with the Mistral 7b model, I see there are no Checkpoints. I've set batch_size to 4 fl16, gradient accumulation to 8,... My JSON file is set on key:value pairs like "instruction" and "output", have changed that where it is "phrase" and "tone". Everything went through just with one Warning but nothing serious. Waited to it finnishes the finetunning. I see models folder with qlora and checkpoints. In QLora I have run_history_gpu-cpu.txt:

There is also another subfolder called again "qlora". Inside are on .json file "gpu-cpu_model.json" and another folder called "gpu-cpu_model". Inside this folder is another folder called adapter with a .json file "adapter_model.json" and "adapter_model.bin"

But in Checkpoints folder there is nothing. As said, tried several times. Different .json datasets. No clue why it doesn't work.

I'm running it on Windows 11 with a NVIDIA RTX 4090. And it shows it is running it on my GPU.

So, what is going on???

When I run python gradio_chat.py I get:

(mistral-7b-env) zasear@zaesarius:/mnt/c/Users/zaesa/OneDrive/Escritorio/AI/Lawyer Mistral Agent/inference$ python gradio_chat.py Number of GPUs available: 1 Running on device: cuda CPU threads: 16 Loading checkpoint shards: 100%|████████████████████████████████████████████| 2/2 [02:44<00:00, 82.49s/it] Traceback (most recent call last): File "/mnt/c/Users/zaesa/OneDrive/Escritorio/AI/Lawyer Mistral Agent/inference/gradio_chat.py", line 40, in <module> usingAdapter = true NameError: name 'true' is not defined

Appreciate any help!

Start by comparing the generated models/qlora contents with the empty Checkpoints folder and review the training warning and run_history_gpu-cpu.txt. Then reproduce the inference command from inference/gradio_chat.py and inspect the traceback at line 40. Done means the cause of the missing checkpoints and the reported inference failure are identified and documented or fixed.