huggingface / huggingface/transformers
Add Flax implementation for BLOOM
- Dominant language
- Python
- Stars
- 166k
- Forks
- 34.6k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 281
Description
### Model description
I'm interested in adding an implementation of BLOOM in Flax.
The implementation shouldn't be too bad since the pytorch implementation can serve as a guide and a way to check correctness.
### Open source status
- [X] The model implementation is available
- [X] The model weights are available
### Provide useful links for the implementation
@younesbelkada @stas00 @patrickvonplaten
If someone is already planning to work on this then no worries, but if not I will start on this as soon as I have time!
Contributor guide
Research direction
Start by reading the existing PyTorch BLOOM implementation and the repository's Flax model implementations for structure and conventions. Use the PyTorch version as the guide and correctness reference; done means a working Flax BLOOM implementation with available weights and behavior matching the reference.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100