huggingface / huggingface/setfit

setfit for ranking problems?

Open
#395 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.8k
Forks
267
Avg merge
36m
Merged PRs (30d)
5

Description

I'd like to finetune setfit for ranking. Where, for each instance, there are possible text anwers in a ranked order.

I'd then like to finetune to maintain the ranking. Would this be possible?

[fast-soft-sort](git@github.com:google-research/fast-soft-sort.git) can be used for differentiable ranking.

The idea might be that I am not fine-tuning the sentence-transformer of the input (or I can, optionally), but the output similarity transformer would be fine-tuned to optimize the rank score.

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named. First determine how SetFit's output similarity transformer is trained and whether fast-soft-sort can provide a differentiable ranking objective; done would require a maintainer-approved design for the training scope and ranking loss.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.