huggingface / huggingface/transformers

Add Flax implementation for BLOOM

Open
#17,703 7 comments 9 reactions 0 assignees View on GitHub
New model
Dominant language
Python
Stars
166k
Forks
34.6k
Avg merge
3d 9h
Merged PRs (30d)
281

Description

### Model description

I'm interested in adding an implementation of BLOOM in Flax.

The implementation shouldn't be too bad since the pytorch implementation can serve as a guide and a way to check correctness.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

@younesbelkada @stas00 @patrickvonplaten

If someone is already planning to work on this then no worries, but if not I will start on this as soon as I have time!

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing PyTorch BLOOM implementation and the repository's Flax model implementations for structure and conventions. Use the PyTorch version as the guide and correctness reference; done means a working Flax BLOOM implementation with available weights and behavior matching the reference.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.