MsaServer GET/template 400 error for colabfold batch use_templates=True

Hi,

We have deployed our own MMSEQS2 server on a p4d.24xlarge AWS instance, running the server with all data residing in memory post vmtouch command and using the Alphafold2_batch.ipynb notebook to send requests against our own server.

We want to run search and predictions while including the use of templates (setting use_templates=True)

Expected Behavior

Output AF2 predictions

Current Behavior

Setting use_templates=True in the run function will result in a 400 bad request to the server

Input example

https://www.rcsb.org/fasta/entry/6MH2

Output

Server logs
"GET /template/1fve_A,6oge_D,7pkl_L,6bi2_M,3n85_L,7kxj_L,2r8s_L,5bo1_M,1tzi_A,7klg_M,5xhv_Q,6u8k_G,1s78_C,3r1g_L,5e08_L,8dp3_L,5xhg_A,1t3f_A,2qqn_L,4kmt_L HTTP/1.1" 400 5639857
Function output logs

Could not get MSA/templates for rcsb_pdb_6MH2: unexpected end of data Traceback (most recent call last): File "/home/ec2-user/SageMaker/localcolabfold/colabfold-conda/lib/python3.10/site-packages/colabfold/batch.py", line 1453, in run = get_msa_and_templates(jobname, query_sequence, a3m_lines, result_dir, msa_mode, use_templates, File "/home/ec2-user/SageMaker/localcolabfold/colabfold-conda/lib/python3.10/site-packages/colabfold/batch.py", line 765, in get_msa_and_templates a3m_lines_mmseqs2, template_paths = run_mmseqs2( File "/home/ec2-user/SageMaker/localcolabfold/colabfold-conda/lib/python3.10/site-packages/colabfold/colabfold.py", line 294, in run_mmseqs2 tar.extractall(path=TMPL_PATH) File "/home/ec2-user/SageMaker/localcolabfold/colabfold-conda/lib/python3.10/tarfile.py", line 2264, in extractall self._extract_one(tarinfo, path, set_attrs=not tarinfo.isdir(), File "/home/ec2-user/SageMaker/localcolabfold/colabfold-conda/lib/python3.10/tarfile.py", line 2327, in _extract_one self._extract_member(tarinfo, os.path.join(path, tarinfo.name), File "/home/ec2-user/SageMaker/localcolabfold/colabfold-conda/lib/python3.10/tarfile.py", line 2410, in _extract_member self.makefile(tarinfo, targetpath) File "/home/ec2-user/SageMaker/localcolabfold/colabfold-conda/lib/python3.10/tarfile.py", line 2463, in makefile copyfileobj(source, target, tarinfo.size, ReadError, bufsize) File "/home/ec2-user/SageMaker/localcolabfold/colabfold-conda/lib/python3.10/tarfile.py", line 254, in copyfileobj raise exception("unexpected end of data") tarfile.ReadError: unexpected end of data 2024-01-23 14:03:21,105 Done

Steps to Reproduce (for bugs)

• Run MMSEQS2 server with 1TB RAM on a SageMaker notebook instance
• Use the Alphafold2_batch.ipynb and point host_url parameter to localhost:80
• add the fasta file in the input directory
• use the mmcif databases as specified for pdbdivided and pdbobsolete

Context

• We are using the following databases:
• uniref30_2302_db: as specified in setup_databases.sh
• colabfold_envdb_202108_db: as specified in setup_databases.sh
• pdb: pdb100_230517 as specified in setup_databases.sh
• pdb70: pdb70_from_mmcif_220313.tar.gzas taken from https://colabfold.mmseqs.com/
• pdbdivided from here : https://files.wwpdb.org/pub/pdb/data/structures/divided/mmCIF/
• pdbobsolete from here: https://files.wwpdb.org/pub/pdb/data/structures/obsolete/mmCIF

Your Environment

• p4d.24xlarge
• Deployed MMSEQS2 server
• Database loaded into memory with vmtouch

Start with the use_templates=True path in colabfold/batch.py and the run_mmseqs2 logic in colabfold/colabfold.py, then reproduce the request from Alphafold2_batch.ipynb against the local server. Check the /template/ request and its response while using the listed databases. Done means template retrieval completes without the 400 response or tarfile.ReadError and AF2 predictions are produced.