huggingface / huggingface/transformers

pointer to transformer (big) model

Open
#16,747 2 comments 0 reactions 0 assignees View on GitHub
New model
Dominant language
Python
Stars
166k
Forks
34.6k
Avg merge
3d 9h
Merged PRs (30d)
281

Description

# 🌟 New model addition

## Model description

Hi, needed a pointer on how to instantiate a Transformer-big from the original Vaswani et. al. paper (Attention Is All You Need). I could only find versions of Transformer-like architectures, so would be useful if this could also be added.

## Open source status

* [x] the model implementation is available: (give details): https://research.google/pubs/pub46201/
* [ ] the model weights are available: (give details)
* [ ] who are the authors: (mention them, if possible by @gh-username)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the linked Vaswani et al. paper and comparing its Transformer-big architecture with the existing Transformer-like implementations in the repository. Done means adding a usable Transformer-big model implementation with a clear instantiation path; the issue does not name specific files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.