After finished fine-tune training.. Invalid header in file issue..!

Hi,

Many thanks for share this wonderful works!

I've trained mistral-7b with lora.py from the instruction and i got a adapters.npz.

I was trying to test this adapter with below commands and issues were raised.

 python lora.py --model /users/soonde17/anaconda3/envs/mlx/mlx/mlx-examples/mlx-examples/mistral-7B-v0.1 --adapter-file /users/soonde17/anaconda3/envs/mlx/mlx/mlx-examples/mlx-examples/lora/adapter_file --test

messages

Traceback (most recent call last):
  File "/Users/soonde17/anaconda3/envs/mlx/mlx/mlx-examples/mlx-examples/lora/lora.py", line 385, in <module>
    model.load_weights(args.adapter_file)
  File "/Users/soonde17/anaconda3/envs/mlx/lib/python3.10/site-packages/mlx/nn/layers/base.py", line 94, in load_weights
    self.update(tree_unflatten(list(mx.load(file).items())))
RuntimeError: [load] Invalid header in file /users/soonde17/anaconda3/envs/mlx/mlx/mlx-examples/mlx-examples/lora/adapter_file

Do i have to fix something when i fine-tune the model with lora? or retry with increase the number of iters?
(I've trained 300 inters for that)

Please help to figure this out..!

Thank you.

Start with lora.py, especially the adapter loading path around the reported traceback, and compare the generated adapters.npz with the adapter_file path passed to --adapter-file. Reproduce the --test command and determine what causes the invalid header; done means the trained adapter loads successfully without that error.