sokrypton / sokrypton/ColabFold
illegal memory access error with GPU notebook
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 2.9k
- Forks
- 747
- PR merge metrics
- No merged PRs in 30d
Description
Expected Behavior
Runs succeed
Current Behavior
Runs fail with an illegal memory access error
Steps to Reproduce (for bugs)
- connect to GPU backend on https://colab.research.google.com/github/sokrypton/ColabFold/blob/main/batch/AlphaFold2_batch.ipynb
- do runs with use_amber & use_templates (unsure if this matters)
ColabFold Output (for bugs)
All sequences (even short ones) give output like this:
2023-02-14 14:09:44,037 Query 35/59: 7s7icomplexColSep (length 329)
2023-02-14 14:10:16,352 Sequence 0 found templates: ['1hyr_C', '1k5n_A', '7lfj_D', '7lfm_D', '1nez_A', '3nfn_A', '7mja_A', '1je6_A', '7jyv_A', '6m2k_A', '1zs8_E', '1c16_G', '1r3h_C', '6at5_A', '7edo_A', '6eny_F', '1s7v_D', '6ddm_C', '6ddv_C']
2023-02-14 14:11:20,929 Sequence 1 found templates: ['7km6_H', '6wzn_A', '3zhk_B', '1igt_B', '1oaq_H', '1fgv_H', '5grv_L', '7cu5_A', '6kyz_B', '5gs1_N', '7n8h_C', '5grw_B', '7orb_A', '1hez_B', '1dee_D', '6wzn_A', '2kh2_B', '4rav_C', '7mhy_O', '2kh2_B']
2023-02-14 14:11:53,738 Sequence 2 found templates: ['6j9o_L', '7jo8_B', '4kv5_E', '4hjj_L', '6tcs_A', '1eeq_B', '6ehv_X', '5gry_G', '7bep_I', '7ekb_L', '5ujz_G', '7d0c_F', '5ujz_G', '6tcs_A', '4k07_H', '5grw_B', '3qrg_L', '5gs1_M', '5gry_G', '7vyt_C']
2023-02-14 14:11:57,850 Padding length to 339
2023-02-14 14:13:02,945 Could not predict 7s7icomplexColSep. Not Enough GPU memory? INTERNAL: Failed to launch CUDA kernel: fusion_862 with block dimensions: 32x1x1 and grid dimensions: 5264x1x1: CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with batch/AlphaFold2_batch.ipynb from the linked Colab reproduction and run the reported GPU case with use_amber and use_templates. Investigate the CUDA_ERROR_ILLEGAL_ADDRESS failure after template discovery and padding; done means the affected runs complete successfully without the illegal memory access error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook
- Domain
- bioinformatics, machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100