allenai / allenai/hyperdecoders
Using Separate Hypernetworks for Each Transformer Layer
Open
- Dominant language
- Python
- Stars
- 14
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
I noticed in the paper that a single hypernetwork is used to generate parameters for all layers. Have you tried using a separate hypernetwork for each Transformer layer? Would this lead to a larger performance improvement?
Thanks!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.