tensorflow / tensorflow/models
Documentation: ELECTRA Pretraining
Open
@malla456 is already working on this.
Since Sep 10, 2026.
models:official
stale
stat:awaiting response
type:docs
- Dominant language
- Python
- Stars
- 77.7k
- Forks
- 44.8k
- PR merge metrics
- No merged PRs in 30d
Description
Dear TF Models Garden Maintainer,
today I recognized that TensorFlow 1.15 is no longer available as supported TPU version. That means, it is no longer possible to pretrain BERT and ELECTRA models with the official code bases.
As I was able to sucessfully pretrain Token Dropping BERT and TEAMS models (many thanks for open sourcing the implementation), I would like to know if it's already possible to pretrain "normal" ELECTRA models with TF Models Garden and if the necessary pretraining commands can be provided here and in the documentation.
Many thanks in advance!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.