facebookresearch / facebookresearch/sam2

"cudnnGetLibConfig" symbol not found error during SAM2ImagePredictor.predict

Open
#670 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
19.9k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Description

I recently installed SAM-2 on a Windows computer with an NVIDIA RTX 3090 GPU, CUDA 12.1 and CUDNN 9.10 installed, and PyTorch 2.5.1 (with CUDA 12.1 support). I am trying to run SAM-2 python code that I have used on a different computer with a different GPU (laptop RTX 3050) and slightly different versions of CUDA, CUDNN, etc.

I am able to load a SAM2 model, but when I use the SAM2ImagePredictor, I get an error stating that it "could not load symbol cudnnGetLibConfig". I believe this error comes from a torch `F.scaled_dot_product_attention()` call in the `Attention()` class in the file transformer.py.
Torch seems to be able to identify a CUDNN installation (`torch.backends.cudnn.version() == 90100`), and also indicates that Flash and CUDNN are enabled for sdp use (`torch.backends.cuda.fash_sdp_enabled() == True` and `torch.backends.cuda.cudn_sdp_enabled() == True`).
I can force the predictor to work by disabling the CUDNN backend (`torch.backends.cudnn.enabled = False`, `torch.backends.cuda.enable_cudnn_sdp(False)`) and changing the `sdp_kernel_context` function in transformer.py to force it to use something else (`torch.backends.cuda.sdp_kernel` call with arguments `enable_mem_efficient=True` and `enable_cudnn=False`).

I'm not familiar enough with torch to determine what kind of attention is actually being used, but it seems like my new installation can't manage to use Flash or CUDNN attentions, and errors out when it tried to actually use/find CUDNN. Is there something I need to set in my PATH or during installation?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.