huggingface / huggingface/peft
Proposal: Benchmarking on different PEFT techniques
- Dominant language
- Python
- Stars
- 21.7k
- Forks
- 2.5k
- Avg merge
- 4d 12h
- Merged PRs (30d)
- 59
Description
I recently read this Hugging Face blog: [**"Beyond LoRA: Can You Beat the Most Popular Fine-Tuning Technique?"**
](https://huggingface.co/blog/peft-beyond-lora)
The blog concludes that, depending on the downstream task, PEFT techniques other than LoRA can outperform it. Inspired by this benchmark, I'd like to contribute by evaluating different PEFT methods on **Table Recognition** and **OCR** tasks using datasets available on Hugging Face.
I plan to run a comprehensive set of PEFT experiments on OCR-focused vision-language models such as:
* `zai-org/GLM-OCR`
* `PaddlePaddle/PaddleOCR-VL-1.6`
* and other relevant OCR/Table Recognition models.
Example dataset: [apoidea/fintabnet-html](https://huggingface.co/datasets/apoidea/fintabnet-html)
The goal is to benchmark different PEFT methods across these tasks and compare their effectiveness in terms of accuracy, efficiency, and resource usage.
If this sounds like a valuable addition to the benchmark, I'd be happy to take ownership of this task. Please assign it to me if you think it would be a good fit.
Contributor guide
Research direction
Start with the linked Hugging Face blog, the GLM-OCR and PaddleOCR-VL-1.6 model resources, and the apoidea/fintabnet-html dataset. Define the benchmark scope and reproducible experiments for OCR and table recognition, then compare PEFT methods by accuracy, efficiency, and resource usage; completion means the results support a documented comparison.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100