Set Transformer

Nobody has claimed this yet.

### Model description

This issue proposes addition of the Set Transformer, a set2seq transformer for learning to order sets of items.

## Short description of the model and link to the paper

The transformer is one of a family of models that implements permutation invariance in order to learn ordering relations. This particular implementation uses stacked attention blocks to achieve the invariance. Set transformers are good for a multitude of problems - the toy problem is the TSP, where vertices are ordered optimally, though the framing can also be applied to any sequence generation tasks where the sequence items are known ahead of time. See [this review](https://jair.org/index.php/jair/article/view/12839) for a description of the family of problems.

This particular transformer is the Set Transformer, presented in [Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks](http://proceedings.mlr.press/v97/lee19d.html).

This isn't immediately designed for text or images or speech, but is a distinct transformer architecture that has been applied to text and image data.

## Link to the implementation if it is open-source

There's an official PyTorch implementation at [https://github.com/juho-lee/set_transformer](https://github.com/juho-lee/set_transformer)

We've already got this up & running as a baseline in an upcoming IJCAI paper

## Link to the model weights if they are available.

not immediately available, but we could work something out

### Open source status

- [x] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

Me & a colleague can get this up onto HF, we have a running implementation and the reference implementation is both on github and licensed MIT. Reference implementation by @juho-lee (author) and @yoonholee.

The issue does not name repository files, tests, or an entry point. Start by reviewing the linked official PyTorch implementation and the cited Set Transformer paper, then compare existing model integrations in Transformers. Done would require a maintained Set Transformer implementation with appropriate validation and documentation, but the issue does not define those acceptance details.

Written by the indexing model from the issue text.