Full parameter training is more easy overfitting than Lora training ？

Nobody has claimed this yet.

Thank you for your excellent work.

We use your code to training Tool selection Task. We tell LLM, there are a list of tools, please select the most relevant tool to answer the Query.

We have the conclusion:
1. Held-in Tools: Full parameter training is better the Lora training; acc=95% vs 91%；
2. Held-out Tools: Full parameter training is worse, acc=78% vs 87%;
3. For a new tools, when using Lora, we just optimize the description, and it reach an acceptable result. But we never reproduce such result by full parameter training, it looks like the LLM has only remember the trained tools, and lacks of ability to generalize to a new tool.

Full parameter setting:
epoch=3
dataset=4W
batch=128

I found no clue about this phenomenon. If you have any experiments, please share with us. We sincerely thank you for your help.

The issue provides full-parameter training settings of epoch=3, dataset=4W, and batch=128, plus held-in and held-out tool accuracy comparisons with LoRA. No files, tests, or entry points are named; begin by identifying the training and evaluation entry points needed to reproduce both settings, and consider the work done when the reported generalization difference is reproduced or explained.

Written by the indexing model from the issue text.