deepspeedai/DeepSpeedExamples

View on GitHub

Example models using DeepSpeed

Stars: 6.8k
Forks: 1.1k
Open beginner issues: 0
Indexed issues: 300
Avg merge: 2d 16h
Merged PRs (30d): 1

Dominant language: Python
License: Apache-2.0
Last GitHub push: Aug 26, 2026
Latest indexed: Sep 20, 2026
Contributing guide: No contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

300 open issues indexed

Why is my model bigger after compression? Open

Difficulty 4/5 3-5 days Newbie friendliness 28/100

deepspeedai/DeepSpeedExamples#209 · 1 comment · Opened Oct 20, 2022
The example of bert compression did not change line 383 in modeling_bert.py? Open

Difficulty 3/5 1-2 days Newbie friendliness 38/100

deepspeedai/DeepSpeedExamples#203 · Opened Sep 28, 2022
XTC in DeepSpeed Compression does not work Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

deepspeedai/DeepSpeedExamples#202 · 1 comment · Opened Sep 27, 2022
Bubbles in Pipeline Parallelism Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

deepspeedai/DeepSpeedExamples#201 · Opened Sep 26, 2022
Failed to run model compression example script for GPT2 in Google Colab Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#199 · 4 comments · Opened Sep 6, 2022
[Question]a bout compression.helper.convert_conv1d_to_linear Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

deepspeedai/DeepSpeedExamples#198 · Opened Sep 6, 2022
ZeRO 3 not working with GAN example: IndexError: tuple index out of range Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#193 · Opened Aug 1, 2022
Is GPU throughput reasonable? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#192 · 3 comments · Opened Aug 1, 2022
BingBertSQuAD Fine-tuning result mismatch tutorial document Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#190 · 1 comment · Opened Jul 28, 2022
OOM despite ZeRO stage 3 Open

Difficulty 4/5 3-5 days Newbie friendliness 32/100

deepspeedai/DeepSpeedExamples#186 · 1 comment · Opened Jul 13, 2022
Megatron-LM-v1.1.5-3D_parallelism ds_pretrain_gpt2_pipe.sh error Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#185 · 5 comments · Opened Jul 6, 2022
My deepspeed code is very slow Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#172 · 30 comments · Opened Mar 27, 2022
'gamma', 'theta' not found in progressive layer drop Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

deepspeedai/DeepSpeedExamples#169 · 3 comments · Opened Mar 14, 2022
deepspeed_aio_handle_t::_stop_threads(): Assertion `0 == _num_pending_ops' failed. Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#167 · Opened Mar 6, 2022
Unable to run inference example Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#165 · 2 comments · Opened Mar 2, 2022
Is there any example with recent version of Megatron-LM? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#162 · 2 comments · Opened Feb 15, 2022
HelloDeepSpeed not reproducible Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

deepspeedai/DeepSpeedExamples#159 · 2 comments · Opened Feb 6, 2022
unable to prodcude bing_bert with nvidia data Open

deepspeedai/DeepSpeedExamples#155 · 11 comments · 1 assignee · Opened Jan 8, 2022
error running DeepSpeedExamples/BingBertSquad/run_squad_deepspeed.sh Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

deepspeedai/DeepSpeedExamples#149 · 1 comment · Opened Nov 23, 2021
Error encountered running DeepSpeedExamples/HelloDeepSpeed/train_bert.py Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

deepspeedai/DeepSpeedExamples#148 · 3 comments · Opened Nov 23, 2021
bing_bert script error Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#147 · 4 comments · Opened Nov 21, 2021
Predict latency is more with 4 GPUs than 1 GPU Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#135 · 1 comment · Opened Oct 5, 2021
Dynamic batch support Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#134 · 1 comment · Opened Sep 15, 2021
How to use Deepspeed checkpoints for BingBertSquad finetuning? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#129 · 2 comments · Opened Aug 19, 2021
Zero shot classification inference Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

deepspeedai/DeepSpeedExamples#128 · Opened Aug 19, 2021
How to enable Model Parallel in bing_bert? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

deepspeedai/DeepSpeedExamples#124 · Opened Aug 12, 2021
Why ZeRO-2 use more CUDA Memory than ZeRO-1？ Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#123 · 1 comment · Opened Aug 12, 2021
"ds_train_bert_nvidia_data_bsz64k_seq128.sh" program stalls at the end of the first epoch Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#122 · 2 comments · Opened Aug 5, 2021
Question: update preprocessing scripts to use HuggingFace datasets for pretraining? Open

Difficulty 4/5 3-5 days Newbie friendliness 32/100

deepspeedai/DeepSpeedExamples#120 · 12 comments · 2 reactions · Opened Aug 3, 2021
Megatron-LM-v1.1.5-ZeRO3: evaluation during training disabled Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

deepspeedai/DeepSpeedExamples#119 · Opened Jul 27, 2021
Does Megatron-LM-v1.1.5-ZeRO3 support 3D parallelism? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#118 · Opened Jul 21, 2021
ascii' codec can't decode byte Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#115 · Opened Jul 15, 2021
Cannot use ZeRO in bing_bert example Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#113 · 2 comments · Opened Jun 21, 2021
TypeError: unsupported operand type(s) for *: 'NoneType' and 'int' Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

deepspeedai/DeepSpeedExamples#108 · 2 comments · Opened May 14, 2021
Any benefits while using a single GTX 1080 Ti? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#106 · 1 comment · Opened May 4, 2021
Megatron-LM-v1.1.5-ZeRO-2/3 doesn't support model change to Huggingface Bert Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#104 · Opened Apr 27, 2021
BERT in Megatron-LM-v1.1.5-3D_parallelism does not support pipeline parallelism Open

Difficulty 4/5 3-5 days Newbie friendliness 32/100

deepspeedai/DeepSpeedExamples#102 · 1 comment · Opened Apr 22, 2021
Which Wikipedia and BookCorpus datasets to use for Bert-pretraining example? Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

deepspeedai/DeepSpeedExamples#101 · 1 comment · Opened Apr 21, 2021
AttributeError: 'Namespace' object has no attribute 'tensor_model_parallel_size' Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

deepspeedai/DeepSpeedExamples#94 · Opened Apr 1, 2021
DeepSpeed not actually reduce wall clock time for 1 epoch - DCGAN Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#87 · Opened Feb 18, 2021
ImportError: No module named 'fused_adam' Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#85 · 3 comments · Opened Feb 14, 2021
RecursionError: maximum recursion depth exceeded while calling a Python object Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

deepspeedai/DeepSpeedExamples#84 · 4 comments · Opened Feb 9, 2021
the question for bing-bert Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#83 · Opened Feb 7, 2021
bing_bert: CUDA error: out of memory Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

deepspeedai/DeepSpeedExamples#76 · 1 comment · Opened Jan 12, 2021
Decay for deepspeed norm & bias params? Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#68 · 2 comments · Opened Dec 15, 2020
Ignore index for mpu cross entropy Open

Difficulty 2/5 1-3 hours Newbie friendliness 42/100

deepspeedai/DeepSpeedExamples#66 · Opened Dec 2, 2020
Any log for bert pre-training? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#65 · 2 comments · Opened Dec 2, 2020
Pre-trained Bert model for BingBertSquad Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

deepspeedai/DeepSpeedExamples#25 · 1 comment · Opened Jun 9, 2020
Megatron-LM BERT Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

deepspeedai/DeepSpeedExamples#18 · Opened Apr 22, 2020
Bing BERT Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

deepspeedai/DeepSpeedExamples#8 · 28 comments · Opened Mar 19, 2020

deepspeedai/DeepSpeedExamples

Get new issues in your inbox