bigscience-workshop / bigscience-workshop/multilingual-modeling
Control Extra Params (use Adapter 16x reduction size as control)
Open
- Dominant language
- Python
- Stars
- 75
- Forks
- 17
- PR merge metrics
- No merged PRs in 30d
Description
The following info is for Bloom-1.3B and embedding-and-MADX-adapters (with replace strategy) with the default bottleneck reduction size of 16.
```
Total frozen parameters: 1208602624
Total trainable parameters: 24979456
Total emb parameters: 20488192
Total MAD-X adapter parameters: 4,491,264
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.