huggingface / huggingface/setfit
Train SBERT using 2 sentences as input for detecting if 2 sentences are duplicates on one another. Using setfit.
- Dominant language
- Jupyter Notebook
- Stars
- 2.8k
- Forks
- 267
- Avg merge
- 36m
- Merged PRs (30d)
- 5
Description
Hi all, I'm trying to train SBERT to classify 2 sentences as being duplicates or not using set fit. How do I make it so that "column_mappings" exepts 2 sentences instead of one?
Below is the code I tried.
#Create trainer
trainer = SetFitTrainer(
model = model, #SBERT model
train_dataset = train_dataset,
eval_dataset = eval_dataset,
loss_class = CosineSimilarityLoss,
metric = "accuracy",
batch_size = 32, #2X num_samples
num_iterations = 60,
num_epochs = 3,
column_mapping ={"sentence1Title": "text1","sentence2Title": "text2", "duplicate": "label", "text": "text"}
)
This is the error I get:
ValueError: The column mapping expected the columns ['duplicate', 'sentence1Title', 'sentence2Title', 'text'] in the dataset, but the dataset had the columns ['Unnamed: 0', 'duplicate', 'sentence1Body', 'sentence1Title', 'sentence2Body', 'sentence2Title'].
thank you in advance!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.