seed

In my shell script file, I used the same seed parameter, e.g., seed=1. When I ran the program twice with the same output path, I noticed that the results were different.

First run (same output path, same seed=1):

json
"metrics": {
"max_ca_deviation": 0.17136359214782715,
"n_chainbreaks": 0,
"n_clashing.interresidue_clashes_w_sidechain": 1,
"n_clashing.interresidue_clashes_w_backbone": 0,
"non_loop_fraction": 0.49333333333333335,
"loop_fraction": 0.5066666666666667,
"helix_fraction": 0.0,
"sheet_fraction": 0.49333333333333335,
"num_ss_elements": 5,
"radius_of_gyration": 11.915282883136799,
"alanine_content": 0.05533596837944664,
"glycine_content": 0.08695652173913043,
"num_residues": 253,
"diffused_com": [0.48821455240249634, -0.3333294689655304, -1.1730750799179077]
}
Second run (same output path, same seed=1):

json
"metrics": {
"max_ca_deviation": 0.17136359214782715,
"n_chainbreaks": 0,
"n_clashing.interresidue_clashes_w_sidechain": 0,
"n_clashing.interresidue_clashes_w_backbone": 0,
"non_loop_fraction": 0.49333333333333335,
"loop_fraction": 0.5066666666666667,
"helix_fraction": 0.0,
"sheet_fraction": 0.49333333333333335,
"num_ss_elements": 5,
"radius_of_gyration": 11.86426138744438,
"alanine_content": 0.05533596837944664,
"glycine_content": 0.08695652173913043,
"num_residues": 253,
"diffused_com": [0.6736019253730774, -0.25595101714134216, -1.0129928588867188]
}
Comparing these two files, I found differences in "diffused_com" and "radius_of_gyration".

When I set seed=1 but used different output paths:

First output path:

json
"metrics": {
"max_ca_deviation": 0.17136359214782715,
"n_chainbreaks": 0,
"n_clashing.interresidue_clashes_w_sidechain": 1,
"n_clashing.interresidue_clashes_w_backbone": 0,
"non_loop_fraction": 0.49333333333333335,
"loop_fraction": 0.5066666666666667,
"helix_fraction": 0.0,
"sheet_fraction": 0.49333333333333335,
"num_ss_elements": 5,
"radius_of_gyration": 11.875816641377464,
"alanine_content": 0.07509881422924901,
"glycine_content": 0.08695652173913043,
"num_residues": 253,
"diffused_com": [0.4528803825378418, -0.30150309205055237, -1.0854626893997192]
}
Second output path:

json
"metrics": {
"max_ca_deviation": 0.17136359214782715,
"n_chainbreaks": 0,
"n_clashing.interresidue_clashes_w_sidechain": 0,
"n_clashing.interresidue_clashes_w_backbone": 0,
"non_loop_fraction": 0.48,
"loop_fraction": 0.52,
"helix_fraction": 0.0,
"sheet_fraction": 0.48,
"num_ss_elements": 5,
"radius_of_gyration": 11.800345615154269,
"alanine_content": 0.06324110671936758,
"glycine_content": 0.08695652173913043,
"num_residues": 253,
"diffused_com": [0.7453345656394958, -0.44733840227127075, -1.0878208875656128]
}
In this case, loop_fraction, sheet_fraction, radius_of_gyration, alanine_content, etc., are all different.

My question is: If RFdiffusion3 is supposed to produce reproducible results, is setting the seed in the shell script file the correct way to do it? Why do I get different results even with the same seed value?Thank you.

Start by reproducing the shell-script runs described in the issue with seed=1, both with the same output path and with different output paths. Compare the reported metrics and trace how the seed and output path are used; done means identifying why results differ or documenting the conditions required for reproducibility.