RF3 training cutoff uses PDB deposition date — implications for temporal benchmark leakage

Hi RF3 / Foundry team,

I am looking into temporal benchmark construction across structure-prediction models and wanted to confirm the exact semantics of the RF3 training cutoff, particularly with respect to possible training/test leakage.

The RF3 README currently lists checkpoints with training cutoffs such as:

Latest: 01/24
Preprint: 01/24
Benchmark: 09/21

From the public training configuration, these appear to be specifically PDB deposition-date cutoffs, rather than public release-date cutoffs.

For example, the RF3 training dataset configuration contains:

filters:

• "deposition_date < '2024-01-01'"

and AtomWorks explicitly distinguishes:

deposition_date
release_date

with deposition_date coming from:

_pdbx_database_status.recvd_initial_deposition_date

while release_date is derived from the earliest:

_pdbx_audit_revision_history.revision_date

Could you confirm that the RF3 01/24 training cutoff should therefore be interpreted as:

initial PDB deposition date < 2024-01-01

rather than:

initial public PDB release date < 2024-01-01

and similarly that the 09/21 benchmark checkpoint uses deposition-date semantics?

There is an important benchmark implication here.

If an external benchmark defines a target as post-training based on its PDB public release date, but that structure was deposited before RF3's training cutoff, then the target could have been eligible for the RF3 training set despite appearing to be post-cutoff under a release-date split.

In that case, temporal evaluations could contain training/test leakage.

This seems potentially relevant to benchmarks such as FoldBench and also to RF3's own benchmark/post-cutoff evaluations. FoldBench explicitly treats training-date overlap as a potential leakage concern, so it would be useful to know whether all RF3 evaluations against such datasets have been checked using deposition date, consistent with the RF3 training filters, rather than only public release date.

Could you clarify:

Is deposition_date indeed the authoritative temporal inclusion criterion for RF3's PDB training data?
Were RF3's internal post-cutoff/benchmark sets filtered using deposition date as well?
For external evaluations such as FoldBench, were targets deposited before the relevant RF3 cutoff excluded or separately identified?
Are there any other training-data sources for which a different temporal criterion was used?

If deposition date is indeed the intended definition, I think it would be helpful for the README to say explicitly:

Training cutoff (PDB initial deposition date): 2024-01-01

rather than only Training Cutoff: 01/24.

That would make contamination checks reproducible and prevent benchmark authors from accidentally treating a release-date split as equivalent to RF3's training cutoff.

Thanks for confirming.