huggingface / huggingface/accelerate

accelerate autocast on mps device

Open
#1,396 9 comments 1 reaction 0 assignees View on GitHub
feature request
Dominant language
Python
Stars
9.9k
Forks
1.5k
Avg merge
5d 2h
Merged PRs (30d)
27

Description

### System Info

```Shell
accelerate==0.18.0
system==M2 macos 13.3.1
```

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] One of the scripts in the examples/ folder of Accelerate or an officially supported `no_trainer` script in the `examples` folder of the `transformers` repo (such as `run_no_trainer_glue.py`)
- [X] My own task or dataset (give details below)

### Reproduction

Down below is the `autocast` implement. The default for fp16 is cuda, I wonder is it possible to adapt on macos mps device. Thanks very much.

```
def autocast(self):
"""
Will apply automatic mixed-precision inside the block inside this context manager, if it is enabled. Nothing
different will happen otherwise.

Example:

```python
>>> from accelerate import Accelerator

>>> accelerator = Accelerator(mixed_precision="fp16")
>>> with accelerator.autocast():
... train()
```
"""
if self.native_amp:
if self.mixed_precision == "fp16" and is_torch_version(">=", "1.10"):
autocast_context = torch.cuda.amp.autocast(dtype=torch.float16)
elif self.mixed_precision == "bf16":
if self.distributed_type in [DistributedType.NO, DistributedType.MULTI_CPU, DistributedType.MULTI_GPU]:
autocast_context = torch.autocast(dtype=torch.bfloat16, device_type=self.device.type)
else:
autocast_context = torch.cuda.amp.autocast()

autocast_context.__enter__()
yield
autocast_context.__exit__(*sys.exc_info())
else:
yield
```

### Expected behavior

Omission

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.