huggingface / huggingface/model-evaluator

Extend support for the zero-shot text classification task to support encoder-decoder models

Open
#70 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
14
Forks
8
Avg merge
34m
Merged PRs (30d)
1

Description

Encoder-decoder models can _also_ be used for pure text generation (e.g. Google's FLAN-T5 series of models), and we should support them in the zero-shot text classification task.

Right now we filter supported models via the 'text-generation' tag on the Hub; we'd need to support the 'text2text generation' tag and in these cases load the model with `AutoModelForSeq2SeqLM` instead of `AutoModelForCausalLM`. From the user perspective this should be invisible — the difference between seq2seq and text generation is blurred since encoder-decoder models do both. It means that the zero-shot task will map to multiple HF tasks.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.