huggingface/accelerate

View on GitHub

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

Stars: 9.9k
Forks: 1.5k
Open beginner issues: 0
Indexed issues: 68
Avg merge: 5d 2h
Merged PRs (30d): 27

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 14, 2026
Latest indexed: Sep 17, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: good first issue

Open All

68 open issues indexed

IterableDataset with batch samples of different size Open

feature request

huggingface/accelerate#348 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Feature request: FSDP for TPUs Open

feature request TPU

huggingface/accelerate#422 · 7 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
XLA + Gradient Accumulation does not work Open

bug feature request TPU

huggingface/accelerate#480 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
I have a problem related to multiprocessing when run accelerate Open

bug TPU wip

huggingface/accelerate#525 · 6 comments · 2 reactions · 1 assignee · Indexed Sep 17, 2026
Feature Contribution Request: neptune.ai and layer.ai logger support Open

feature request

huggingface/accelerate#560 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Feature Request: Context manager for downcasting on TPUs Open

feature request TPU

huggingface/accelerate#587 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Multi-Node Training No progress on Azure Open

bug feature request

huggingface/accelerate#694 · 13 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[RFC]Integrate Intel Extension for Pytorch into accelerate to get out-of-box optimizations on intel platform. Open

feature request

huggingface/accelerate#700 · 9 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
Not possible to use the notebook_launcher on a cluster of A6000 series Open

bug feature request

huggingface/accelerate#721 · 22 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Tutorial] Distributed inference in a notebook Open

documentation feature request

huggingface/accelerate#787 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
More APIs for high-level library development Open

enhancement feature request

huggingface/accelerate#852 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
CUDA initialization Open

wip

huggingface/accelerate#908 · 21 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Need to do distributed training using 2 separate machines Open

enhancement feature request

huggingface/accelerate#924 · 2 comments · 1 reaction · 1 assignee · Indexed Sep 17, 2026
[feature request] A more flexible `free_memory` function Open

enhancement feature request

huggingface/accelerate#925 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
`Accelerate test` failing on an 8-GPU docker instance Open

bug wip

huggingface/accelerate#934 · 6 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
accelerator.end_training() is generating exception when wandb is being used as tracker Open

bug wip

huggingface/accelerate#944 · 15 comments · 5 reactions · 1 assignee · Indexed Sep 17, 2026
Be able to use `notebook_launcher` with TPUs on cloud TPU instances Open

enhancement TPU wip

huggingface/accelerate#967 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Different accelerate lauch behavior via ssh command Open

bug TPU wip

huggingface/accelerate#987 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[feature request] implement `set_verbosity` Open

enhancement feature request

huggingface/accelerate#1066 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
how to cleanly exit when using `accelerate launch` Open

enhancement feature request

huggingface/accelerate#1089 · 25 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
webdataset compatibility / example Open

documentation wip

huggingface/accelerate#1092 · 1 comment · 2 reactions · 1 assignee · Indexed Sep 17, 2026
Add backend to InitProcessGroupKwargs Open

enhancement feature request

huggingface/accelerate#1105 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[feature request] INT4 inference support and flexgen like offloading. Open

feature request

huggingface/accelerate#1157 · 0 comments · 10 reactions · 0 assignees · Indexed Sep 17, 2026
Feature request - SLURM support Open

documentation enhancement feature request

huggingface/accelerate#1239 · 23 comments · 39 reactions · 0 assignees · Indexed Sep 17, 2026
[Feature Request] Publish `accelerate` conda package on the `huggingface` channel Open

wip

huggingface/accelerate#1357 · 1 comment · 1 reaction · 1 assignee · Indexed Sep 17, 2026
Model Parallelism and accelerate's usage of DDP aren't compatible Open

enhancement feature request

huggingface/accelerate#1368 · 18 comments · 3 reactions · 0 assignees · Indexed Sep 17, 2026
[Feature Request] Support Megatron-Deepspeed Open

enhancement feature request

huggingface/accelerate#1384 · 5 comments · 3 reactions · 0 assignees · Indexed Sep 17, 2026
accelerate autocast on mps device Open

feature request

huggingface/accelerate#1396 · 9 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
Accumulate the gradients of 2 models Open

enhancement feature request

huggingface/accelerate#1414 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Feature]: CPU+GPU using accelerate? Open

feature request

huggingface/accelerate#1421 · 8 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Accelerate - Vertex AI integration Open

enhancement feature request

huggingface/accelerate#1459 · 1 comment · 2 reactions · 0 assignees · Indexed Sep 17, 2026
Support Amazon SageMaker Inputs other than S3 Open

enhancement feature request

huggingface/accelerate#1485 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
HuggingFace Accelerate Support for Gaudi Open

enhancement feature request

huggingface/accelerate#1487 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Can accelerate train a single model on multiple TPU VMs (not a TPU Pod)? Open

feature request TPU

huggingface/accelerate#1509 · 7 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Feature request: Slurm env detection Open

enhancement feature request

huggingface/accelerate#1811 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Feature Request] support to pass custom slice function in accelerate data loader (useful for models like Flamingo) Open

enhancement feature request

huggingface/accelerate#1816 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Title: Support for non-tensor based datasets in Accelerate Open

enhancement feature request

huggingface/accelerate#1878 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Feature request: support for multi-node, multi-GPU distributed inference Open

enhancement feature request

huggingface/accelerate#1890 · 8 comments · 20 reactions · 0 assignees · Indexed Sep 17, 2026
Feature request: ability to truncate datasets on split rather than pad Open

enhancement feature request

huggingface/accelerate#2019 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Loading weights straight to GPU only works for inference Open

enhancement feature request

huggingface/accelerate#2028 · 8 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[FSDP] support activation offloading with FSDP Open

enhancement feature request

huggingface/accelerate#2038 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
WebLoader Support Open

enhancement feature request

huggingface/accelerate#2083 · 9 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
stderr: AssertionError: Main process was not first Open

wip

huggingface/accelerate#2084 · 24 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Accelerate integration with Transformer Engine crashes when using FlashAttention Open

enhancement feature request

huggingface/accelerate#2154 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[feature-request] Add OpenVINO as an inference-only backend Open

feature request

huggingface/accelerate#2306 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Why does accelerate's MegatronLMPlugin use its own megatron? Open

wip

huggingface/accelerate#2390 · 19 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[feature request] making the use of the `accelerate` launcher optional Open

enhancement feature request

huggingface/accelerate#2585 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
`torch.compile` not working with `device_map` and multiple GPUs Open

big model inference wip

huggingface/accelerate#2592 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
gather objects in TPU is not supported Open

feature request TPU

huggingface/accelerate#2858 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Dataloader WeightedRandomSampler + Distributed Training Open

enhancement feature request

huggingface/accelerate#2865 · 4 comments · 2 reactions · 0 assignees · Indexed Sep 17, 2026

huggingface/accelerate

Get new issues in your inbox