InseeFrLab / InseeFrLab/torchTextClassifiers

Issue in reproducibility

Open
#95 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
23
Forks
7
PR merge metrics
No merged PRs in 30d

Description

I am unable to get the exact result after training a ttc model on my data (synthetic data), even after setting every possible seed:
```{python}
torch.manual_seed(42)
random.seed(42)
np.random.seed(42)

torch.use_deterministic_algorithms(True)
if torch.cuda.is_available():
torch.cuda.manual_seed_all(42)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False

pytorch_lightning.seed_everything(42)
```

For full reproducibility:

```{bash}
git clone -b main https://github.com/mateomorin/ttc-training.git
# uncomment last test of test_reproducibility
# potentialy change your data path if you do not have access to bucket projet-ape
uv run pytest # should have 2 passed, 1 failed and potential warnings
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Clone the main branch of mateomorin/ttc-training, uncomment the last test in test_reproducibility, and run uv run pytest with the provided seeding setup. Compare the reproducibility test results and warnings; done should mean the test no longer demonstrates differing results after training on the synthetic data.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.