explosion / explosion/curated-transformers

Support for Encoder-Decoder-style architectures

Open
#340 2 comments 3 reactions 0 assignees View on GitHub
feat/model type/feature
Dominant language
Python
Stars
890
Forks
36
PR merge metrics
No merged PRs in 30d

Description

I regularly follow the developments on this project, and I must say that I am very interested and pleased with the direction `curated-transformers` is taking. The code is very understandable and high-quality, it's a pleasure to work with, congratulations!

This is perhaps already in your plans, but just to mention it here, I think a very nice addition to the project would be to have at least one reference implementation of an encoder-decoder style Transformers, such as the T5 architecture. T5 models are very popular for some tasks, especially in the < 1B parameters range which is still very relevant nowadays. Currently we have reference implementations for decoder-style and encoder-style models, but we're missing at least one reference implementation of an encoder-decoder-style architecture, perhaps with a reusable cross-attention block.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.