huggingface / huggingface/setfit

`max_length` parameter of `TrainingArguments` not applied

Open
#561 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.8k
Forks
267
Avg merge
36m
Merged PRs (30d)
5

Description

**Description:**
I don't believe the [`max_length` parameter](https://huggingface.co/docs/setfit/en/reference/trainer#setfit.TrainingArguments.max_length) of `TrainingArguments` is actually being used.

Minimal working example (borrowed from [quickstart](https://huggingface.co/docs/setfit/en/quickstart)):
```
from datasets import load_dataset
from setfit import SetFitModel, Trainer, TrainingArguments, sample_dataset

model = SetFitModel.from_pretrained("BAAI/bge-small-en-v1.5")
dataset = load_dataset("SetFit/sst2")
train_dataset = sample_dataset(dataset["train"], label_column="label", num_samples=8)
model.labels = ["negative", "positive"]

args = TrainingArguments(
max_length=5,
batch_size=32,
num_epochs=10,
)

trainer = Trainer(
model=model,
args=args,
train_dataset=train_dataset,
)
trainer.train()
```

**Expected Behavior:**
- Batches with max_length of 5 in the Transformers training loop

**Actual Behavior:**
- Batches whose # of tokens is equal to the longest token length in the batch

**Possible Culprits:**
- [This function](https://github.com/huggingface/setfit/blob/146c7c9dacdc7dca678b2fffff8ddeb79dd762c2/src/setfit/trainer.py#L116) is responsible for propagating SetFit training arguments to the SentenceTransformer Trainer. `args.max_length` is not referenced in this method, nor does it look like it's supported anyway as a parameter to the [Trainer](https://sbert.net/docs/package_reference/sentence_transformer/trainer.html#sentencetransformertrainer)

**Environment Info:**
```
{'python': '3.11.7',
'sentence_transformers': '3.1.1',
'transformers': '4.44.2',
'torch': '2.4.1',
'accelerate': '0.34.2',
'datasets': '3.0.0',
'tokenizers': '0.19.1'}
```

Happy to look into / propose a fix if appropriate!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.