RosettaCommons / RosettaCommons/foundry
[Bug] RF3 validation step in end-to-end notebook uses undesigned RFD3 backbone instead of MPNN-designed structure, causing abnormally high RMSD
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 966
- Forks
- 181
- Avg merge
- 4d 4h
- Merged PRs (30d)
- 2
Description
Description:
While running the official end-to-end notebook (RFD3 → MPNN → RF3), the RF3 validation step consistently produces abnormally high backbone RMSD (10.56–38.19 Å, expected < 2 Å for successful designs), along with low confidence metrics .
Reproduction / Diagnosis:
I decomposed the pipeline into three independent scripts to isolate the issue:
Section 1 (RFD3 backbone generation): verified normal — 80-residue single chain with expected secondary structure
Section 2 (MPNN sequence design): verified normal — output sequence length and amino acid distribution as expected
Section 3 (RF3 validation): RMSD abnormal — issue isolated here Tracing variable usage in Section 3:
Original notebook code
input_structure = InferenceInput.from_atom_array(
atom_array, example_id="example_protein"
)
...
aa_generated = atom_array # Original RFD3 backbone (Section 1)
aa_refolded = rf3_output.atom_array # RF3-predicted structure
Root cause:
The variable atom_array is defined in Section 1 and never reassigned to the MPNN-designed output. As a result, Section 3 feeds the original undesigned RFD3 backbone (without designed sequence) into RF3 for validation, rather than the actual MPNN-designed structure (stored in mpnn_outputs[i].atom_array). Since RF3 predicts structure from sequence, feeding an undesigned sequence naturally produces a structure nearly unrelated to the original backbone, explaining the high RMSD.
# Corrected
designed_structure = mpnn_outputs[i].atom_array
input_structure = InferenceInput.from_atom_array(
designed_structure, example_id=f"design_{i}"
)
After this fix, RMSD returns to normal range (< 1 Å), with pTM > 0.9.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the official end-to-end notebook and inspect Section 3, the RF3 validation step, alongside the MPNN output handling. Ensure validation uses the MPNN-designed structure from mpnn_outputs[i].atom_array, then rerun the notebook and confirm backbone RMSD is below 2 Å with pTM above 0.9.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 78/100