deepseek-ai / deepseek-ai/DeepSeek-VL

Support for M1 Mac, or non-cuda devices

Open
#11 3 comments 2 reactions 0 assignees Claimed by @Fodark View on GitHub
enhancement
Dominant language
Python
Stars
4.2k
Forks
596
PR merge metrics
No merged PRs in 30d

Description

Seems like parts of the code explicitly call cuda functions, so you can't just switch it to MPS for Macs. Any roadmap for supporting Macs?

Io.py
```python
def load_pretrained_model(model_path: str):
vl_chat_processor: VLChatProcessor = VLChatProcessor.from_pretrained(model_path)
tokenizer = vl_chat_processor.tokenizer

vl_gpt: MultiModalityCausalLM = AutoModelForCausalLM.from_pretrained(
model_path, trust_remote_code=True
)
vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval()
```

error

```
File "/Users/user/projects/ai/DeepSeek-VL/deepseek_vl/utils/io.py", line 37, in load_pretrained_model
vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval()
File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/transformers/modeling_utils.py", line 2528, in cuda
return super().cuda(*args, **kwargs)
File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 911, in cuda
return self._apply(lambda t: t.cuda(device))
File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 802, in _apply
module._apply(fn)
File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 802, in _apply
module._apply(fn)
File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 802, in _apply
module._apply(fn)
[Previous line repeated 2 more times]
File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 825, in _apply
param_applied = fn(param)
File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 911, in
return self._apply(lambda t: t.cuda(device))
File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.