huggingface / huggingface/setfit

Train SBERT using 2 sentences as input for detecting if 2 sentences are duplicates on one another. Using setfit.

Open
#382 3 comments 2 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.8k
Forks
267
Avg merge
36m
Merged PRs (30d)
5

Description

Hi all, I'm trying to train SBERT to classify 2 sentences as being duplicates or not using set fit. How do I make it so that "column_mappings" exepts 2 sentences instead of one?

Below is the code I tried.

#Create trainer
trainer = SetFitTrainer(
model = model, #SBERT model
train_dataset = train_dataset,
eval_dataset = eval_dataset,
loss_class = CosineSimilarityLoss,
metric = "accuracy",
batch_size = 32, #2X num_samples
num_iterations = 60,
num_epochs = 3,
column_mapping ={"sentence1Title": "text1","sentence2Title": "text2", "duplicate": "label", "text": "text"}
)

This is the error I get:
ValueError: The column mapping expected the columns ['duplicate', 'sentence1Title', 'sentence2Title', 'text'] in the dataset, but the dataset had the columns ['Unnamed: 0', 'duplicate', 'sentence1Body', 'sentence1Title', 'sentence2Body', 'sentence2Title'].

thank you in advance!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.