Internal input/output error

I'm using AF3 at the end of pipeline after proteinMPNN design to predict the structure of the sequences generated. I have a python script to convert the .fasta output from MPNN to an accepted .json input for AF3. This typically runs well for a while but sometimes at certain files, AF3 gets an error pasted below.

I believe it's an internal error because the file at which it crashes does get loaded and goes through some iterations of MSA. The "LDKNRI..." is the sequence of the file at which it crashes. The previous file 'binder4_partdif_240_sample2' worked well and the files are generated from the same script. In the input, sequence A is the variable sequence while sequence B is the same for all the input files.

Whenever I try to restart a run of AF3 with this as input file, it always fails at this file. Is there something wrong with the sequence or with how AF is run?

Error file:

```
I0807 06:37:57.378821 140572204999232 pipeline.py:115] Getting protein MSAs took 642.57 seconds for sequence LDKNRIEFINKVQAFIDQIDKAFTLVLKAKEEKTEESIEEIIKKTAKEIAKEVQEVVELVLKHPEYLQYLTRLAETQLLARLLQRGEPAELAVCRVIFLVICKLCKLCGWSKVETIEFFLELTWELFPEASIELTLEILEQAEAQGLLSPEEIAELREKIAERRQEEA
I0807 06:37:57.379011 140572204999232 pipeline.py:121] Deduplicating MSAs for sequence LDKNRIEFINKVQAFIDQIDKAFTLVLKAKEEKTEESIEEIIKKTAKEIAKEVQEVVELVLKHPEYLQYLTRLAETQLLARLLQRGEPAELAVCRVIFLVICKLCKLCGWSKVETIEFFLELTWELFPEASIELTLEILEQAEAQGLLSPEEIAELREKIAERRQEEA
I0807 06:37:57.380460 140572204999232 pipeline.py:134] Deduplicating MSAs took 0.00 seconds for sequence LDKNRIEFINKVQAFIDQIDKAFTLVLKAKEEKTEESIEEIIKKTAKEIAKEVQEVVELVLKHPEYLQYLTRLAETQLLARLLQRGEPAELAVCRVIFLVICKLCKLCGWSKVETIEFFLELTWELFPEASIELTLEILEQAEAQGLLSPEEIAELREKIAERRQEEA, found 4 unpaired sequences, 18 paired sequences
I0807 06:37:57.380597 140572204999232 pipeline.py:40] Getting protein templates for sequence LDKNRIEFINKVQAFIDQIDKAFTLVLKAKEEKTEESIEEIIKKTAKEIAKEVQEVVELVLKHPEYLQYLTRLAETQLLARLLQRGEPAELAVCRVIFLVICKLCKLCGWSKVETIEFFLELTWELFPEASIELTLEILEQAEAQGLLSPEEIAELREKIAERRQEEA
I0807 06:37:57.381286 140572204999232 subprocess_utils.py:68] Launching subprocess "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/HMMER/3.4-gompi-2024a/bin/hmmbuild --informat stockholm --hand --amino /tmp/tmpq6su23pp/output.hmm /tmp/tmpq6su23pp/query.msa"
I0807 06:37:57.674645 140572204999232 subprocess_utils.py:97] Finished Hmmbuild in 0.292 seconds
I0807 06:37:57.675832 140572204999232 subprocess_utils.py:68] Launching subprocess "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/HMMER/3.4-gompi-2024a/bin/hmmsearch --noali --cpu 8 --F1 0.1 --F2 0.1 --F3 0.1 -E 100 --incE 100 --domE 100 --incdomE 100 -A /tmp/tmp44hi8ohs/output.sto /tmp/tmp44hi8ohs/query.hmm /mpcdf/soft/eb/data/AlphaFold/DB/3/3.0.1/pdb_seqres_2022_09_28.fasta"
I0807 06:38:01.729863 140572204999232 subprocess_utils.py:97] Finished Hmmsearch in 4.054 seconds
Traceback (most recent call last):
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/AlphaFold3/3.0.1-foss-2024a-CUDA-12.6.0/bin/run_alphafold.py", line 810, in
app.run(main)
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/absl-py/2.1.0-GCCcore-13.3.0/lib/python3.12/site-packages/absl/app.py", line 308, in run
_run_main(main, args)
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/absl-py/2.1.0-GCCcore-13.3.0/lib/python3.12/site-packages/absl/app.py", line 254, in _run_main
sys.exit(main(argv))
^^^^^^^^^^
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/AlphaFold3/3.0.1-foss-2024a-CUDA-12.6.0/bin/run_alphafold.py", line 793, in main
process_fold_input(
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/AlphaFold3/3.0.1-foss-2024a-CUDA-12.6.0/bin/run_alphafold.py", line 620, in process_fold_input
fold_input = pipeline.DataPipeline(data_pipeline_config).process(fold_input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/AlphaFold3/3.0.1-foss-2024a-CUDA-12.6.0/lib/python3.12/site-packages/alphafold3/data/pipeline.py", line 540, in process
processed_chains.append(self.process_protein_chain(chain))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/AlphaFold3/3.0.1-foss-2024a-CUDA-12.6.0/lib/python3.12/site-packages/alphafold3/data/pipeline.py", line 423, in process_protein_chain
unpaired_msa, paired_msa, template_hits = _get_protein_msa_and_templates(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/AlphaFold3/3.0.1-foss-2024a-CUDA-12.6.0/lib/python3.12/site-packages/alphafold3/data/pipeline.py", line 143, in _get_protein_msa_and_templates
protein_templates = _get_protein_templates(
^^^^^^^^^^^^^^^^^^^^^^^
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/AlphaFold3/3.0.1-foss-2024a-CUDA-12.6.0/lib/python3.12/site-packages/alphafold3/data/pipeline.py", line 41, in _get_protein_templates
protein_templates = templates_lib.Templates.from_seq_and_a3m(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/AlphaFold3/3.0.1-foss-2024a-CUDA-12.6.0/lib/python3.12/site-packages/alphafold3/data/templates.py", line 475, in from_seq_and_a3m
return cls.from_hmmsearch_a3m(
^^^^^^^^^^^^^^^^^^^^^^^
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/AlphaFold3/3.0.1-foss-2024a-CUDA-12.6.0/lib/python3.12/site-packages/alphafold3/data/templates.py", line 551, in from_hmmsearch_a3m
hits = _filter_hits(
^^^^^^^^^^^^^
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/AlphaFold3/3.0.1-foss-2024a-CUDA-12.6.0/lib/python3.12/site-packages/alphafold3/data/templates.py", line 374, in _filter_hits
for hit in hits:
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/AlphaFold3/3.0.1-foss-2024a-CUDA-12.6.0/lib/python3.12/site-packages/alphafold3/data/templates.py", line 525, in hit_generator
release_date, sequence, unresolved_res_ids = _parse_hit_metadata(
^^^^^^^^^^^^^^^^^^^^
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/AlphaFold3/3.0.1-foss-2024a-CUDA-12.6.0/lib/python3.12/site-packages/alphafold3/data/templates.py", line 777, in _parse_hit_metadata
cif = mmcif.from_string(structure_store.get_mmcif_str(pdb_id))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/AlphaFold3/3.0.1-foss-2024a-CUDA-12.6.0/lib/python3.12/site-packages/alphafold3/data/structure_stores.py", line 88, in get_mmcif_str
return filepath.read_text()
^^^^^^^^^^^^^^^^^^^^
File "/mpcdf/soft/eb/x86_64/SLES/15/genoa/software/Python/3.12.3-GCCcore-13.3.0/lib/python3.12/pathlib.py", line 1028, in read_text
return f.read()
^^^^^^^^
OSError: [Errno 5] Input/output error
```

log file

```
Predicting 3D structure for binder4_partdif_240_sample2 for seed(s) (1,)...
Featurising data for seeds (1,)...
Featurising binder4_partdif_240_sample2 with rng_seed 1.
Featurising binder4_partdif_240_sample2 with rng_seed 1 took 7.43 seconds.
Featurising data for seeds (1,) took 7.43 seconds.
Running model inference for seed 1...
Running model inference for seed 1 took 24.47 seconds.
Extracting output structures (one per sample) for seed 1...
Extracting output structures (one per sample) for seed 1 took 0.63 seconds.
Running model inference and extracting output structures for seed 1 took 25.11 seconds.
Running model inference and extracting output structures for seeds (1,) took 25.11 seconds.
Writing outputs for binder4_partdif_240_sample2 for seed(s) (1,)...
Done processing fold input binder4_partdif_240_sample2.
Processing fold input #189
Processing fold input binder4_partdif_241_sample1
Checking we can load the model parameters...
Running data pipeline...
Processing chain A
```

input file

```json
{
"name": "binder4_partdif_241_sample1",
"sequences": [
{
"protein": {
"id": [
"A"
],
"sequence": "LDKNRIEFINKVQAFIDQIDKAFTLVLKAKEEKTEESIEEIIKKTAKEIAKEVQEVVELVLKHPEYLQYLTRLAETQLLARLLQRGEPAELAVCRVIFLVICKLCKLCGWSKVETIEFFLELTWELFPEASIELTLEILEQAEAQGLLSPEEIAELREKIAERRQEEA"
}
},
{
"protein": {
"id": "B",
"sequence": "KPVTIISVSHIQLLSQTDEPSDVVKENFKARPYIICGIQGPSHPTLTMPTETKATFGVHLKQDSLCVFWQYPKLYIDGPFGSPFEESLNYETPFILNTLLDDWKPYKLRRLYFIWVCRDIQSFRWFADLENRPDYVNKYQALNSIFRDLLLPPSSRLFIGRPRWKLIAKCNRGKVGTVFCSISKTLHKLSNRTRFEYNKESFS"
}
}
],
"modelSeeds": [
1
],
"dialect": "alphafold3",
"version": 1
}
```

Start in alphafold3/data/templates.py at _parse_hit_metadata and _filter_hits, then inspect alphafold3/data/structure_stores.py at get_mmcif_str. Reproduce the failure with the supplied JSON and determine whether one template hit or the structure database causes the filesystem I/O error. Done means identifying a reproducible root cause and an appropriate tested failure-handling change or documented limitation.

Written by the indexing model from the issue text.