huggingface / huggingface/sentence-transformers

Did you forget to write "loss_model.eval()"?

Open
#162 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
19.1k
Forks
2.9k
Avg merge
1d 19h
Merged PRs (30d)
61

Description

```
def _eval_during_training(self, evaluator, output_path, save_best_model, epoch, steps):
"""Runs evaluation during the training"""
if evaluator is not None:
score = evaluator(self, output_path=output_path, epoch=epoch, steps=steps)
if score > self.best_score and save_best_model:
self.save(output_path)
self.best_score = score
```

The method is called here:
```
if evaluation_steps > 0 and training_steps % evaluation_steps == 0:
self._eval_during_training(evaluator, output_path, save_best_model, epoch, training_steps)
for loss_model in loss_models:
loss_model.zero_grad()
loss_model.train()
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at _eval_during_training and its call site in the training loop shown in the issue. Trace how evaluator and loss_models are used during evaluation, especially each loss_model's train/eval mode, and determine the expected mode before and after evaluation. Done means the evaluation path uses the intended mode without leaving training behavior incorrect.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.