RosettaCommons / RosettaCommons/foundry

[Bug] RF3 validation step in end-to-end notebook uses undesigned RFD3 backbone instead of MPNN-designed structure, causing abnormally high RMSD

Open Beginner friendly
#380 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
966
Forks
181
Avg merge
4d 4h
Merged PRs (30d)
2

Description

Description:

While running the official end-to-end notebook (RFD3 → MPNN → RF3), the RF3 validation step consistently produces abnormally high backbone RMSD (10.56–38.19 Å, expected < 2 Å for successful designs), along with low confidence metrics .

Reproduction / Diagnosis:

I decomposed the pipeline into three independent scripts to isolate the issue:

Section 1 (RFD3 backbone generation): verified normal — 80-residue single chain with expected secondary structure
Section 2 (MPNN sequence design): verified normal — output sequence length and amino acid distribution as expected
Section 3 (RF3 validation): RMSD abnormal — issue isolated here Tracing variable usage in Section 3:

Original notebook code

input_structure = InferenceInput.from_atom_array(
    atom_array, example_id="example_protein"
)
...
aa_generated = atom_array              # Original RFD3 backbone (Section 1)
aa_refolded = rf3_output.atom_array    # RF3-predicted structure

Root cause:

The variable atom_array is defined in Section 1 and never reassigned to the MPNN-designed output. As a result, Section 3 feeds the original undesigned RFD3 backbone (without designed sequence) into RF3 for validation, rather than the actual MPNN-designed structure (stored in mpnn_outputs[i].atom_array). Since RF3 predicts structure from sequence, feeding an undesigned sequence naturally produces a structure nearly unrelated to the original backbone, explaining the high RMSD.

# Corrected
designed_structure = mpnn_outputs[i].atom_array
input_structure = InferenceInput.from_atom_array(
    designed_structure, example_id=f"design_{i}"
)

After this fix, RMSD returns to normal range (< 1 Å), with pTM > 0.9.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the official end-to-end notebook and inspect Section 3, the RF3 validation step, alongside the MPNN output handling. Ensure validation uses the MPNN-designed structure from mpnn_outputs[i].atom_array, then rerun the notebook and confirm backbone RMSD is below 2 Å with pTM above 0.9.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.