bigscience-workshop / bigscience-workshop/t-zero

reproducibility versus replicability

Open
#7 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
463
Forks
53
PR merge metrics
No merged PRs in 30d

Description

Great to see this implemented. Definitions of reproducibility and replication differ across domains, and would probably be helpful to clarify these in a few places. Happy to add these. In probably the most accepted definition now reproducibility would mean the use of the same data sets, techniques, scripts, and framework by independent researchers to obtain the same results. Replication in this setting is a bit tricky though—would the only difference be in the implemented framework, TensorFlow versus PyTorch? Are there other underlying differences between the two frameworks, which may contribute to differences also in how the model is trained or in the results, depending on whether model training or results are replicated?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.