huggingface / huggingface/peft
Proposal: support ShadowPEFT method in PEFT
- Dominant language
- Python
- Stars
- 21.7k
- Forks
- 2.5k
- Avg merge
- 4d 12h
- Merged PRs (30d)
- 59
Description
### Feature request
Dear PEFT team,
I am a big fan of the peft library. Thanks for your contributions to making this project so easy to use.
Recently, we proposed a new PEFT method -- **ShadowPEFT** ([ArXiv](https://arxiv.org/abs/2604.19254) | [HuggingFace](https://huggingface.co/collections/shadow-llm/shadow-peft-models)).
Unlike LoRAs that inject trainable parameters to the base's weights (weight-level), ShadowPEFT centralizes trainable parameters in a shadow network (architecturally similar to the backbone) and injects it at each Transformer layer (layer-level).
The shadow network operates in parallel with the base model, delivering learned corrections to each decoder layer. Because the shadow module is architecturally decoupled from the base, it can be trained, stored, and deployed independently, benefiting edge computing scenarios and edge-cloud collaboration.
We followed PEFT's API to implement ShadowPEFT in [GitHub:ShadowLLM/shadow-peft](https://github.com/ShadowLLM/shadow-peft). We would be thrilled to see ShadowPEFT considered for integration into the official PEFT library.
Best,
Xianming
### Your contribution
- integration test
Contributor guide
Assessment
This issue has not been assessed yet.