Using SetFit Embeddings for Semantic Search?

Hi,

I was wondering if the semantic search would improve if one would train a multilabel-classification model and use those embeddings?

After training a binary classification model I have seen that the embeddings between similar topics on `all-MiniLM-L12-v2` vs `all-MiniLM-L12-v2-setfit` (fitted model) are very close in fitted model which makes sense for me.

```python
# Cosine Similarity
def get_cosine_similarity(vector1, vector2):
sim = 1 - spatial.distance.cosine(vector1, vector2)
return sim

word_1 = "acne"
word_2 = "red skin"

emb_fit_1 = model.model_body.encode([word_1])
emb_fit_2 = model.model_body.encode([word_2])

emb_base_1 = model_sbert.encode([word_1])
emb_base_2 = model_sbert.encode([word_2])

print(f"{word_1} vs {word_2} (base)", get_cosine_similarity(emb_base_1, emb_base_2))
print(f"{word_1} vs {word_2} (fit)", get_cosine_similarity(emb_fit_1, emb_fit_2))
```

```
acne vs pimple (base) 0.5959747433662415
acne vs pimple (fit) 0.9996786117553711

acne vs red skin (base) 0.36421263217926025
acne vs red skin (fit) 0.9994498491287231

acne vs red car (base) 0.17558744549751282
acne vs red car (fit) 0.0051751588471233845
```

I would assume that if the model is trained on multi-label-classification task the embeddings would somehow clustered based on the labels which are provided during training. Would that improve the semantic search if enough labels are provided during training?

Of course I could train a model and test it but maybe you have done similar tests and already know if it's working or not :-)

Thanks!