huggingface / huggingface/transformers
pointer to transformer (big) model
- Dominant language
- Python
- Stars
- 166k
- Forks
- 34.6k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 281
Description
# 🌟 New model addition
## Model description
Hi, needed a pointer on how to instantiate a Transformer-big from the original Vaswani et. al. paper (Attention Is All You Need). I could only find versions of Transformer-like architectures, so would be useful if this could also be added.
## Open source status
* [x] the model implementation is available: (give details): https://research.google/pubs/pub46201/
* [ ] the model weights are available: (give details)
* [ ] who are the authors: (mention them, if possible by @gh-username)
Contributor guide
Research direction
Start by reviewing the linked Vaswani et al. paper and comparing its Transformer-big architecture with the existing Transformer-like implementations in the repository. Done means adding a usable Transformer-big model implementation with a clear instantiation path; the issue does not name specific files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100