databricks/megablocks

View on GitHub

Stars: 1.6k
Forks: 233
Open beginner issues: 0
Indexed issues: 43

Dominant language: Python
License: Apache-2.0
Last GitHub push: Mar 25, 2026
Latest indexed: Sep 15, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: help wanted
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

43 open issues indexed

[RFC] Define accelerator-agnostic backend boundaries for core MoE operations Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

databricks/megablocks#188 · 2 comments · Opened Sep 7, 2026
Energy Efficiency: 10 Mathematical Techniques for 60-70% AI Energy Reduction (Phi6Simple, FFT-Mix, Phi MoE) Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

databricks/megablocks#184 · Opened Mar 27, 2026
Compatibility with CUDA 13.0 Open

Difficulty 3/5 1-2 days Newbie friendliness 30/100

databricks/megablocks#180 · 1 comment · Opened Oct 23, 2025
difference between cuda kernel code and using torch.ops Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

databricks/megablocks#179 · Opened Sep 22, 2025
Issue with Sparse MLP Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

databricks/megablocks#178 · 1 comment · Opened Aug 14, 2025
MOE uses more memory than dense model and is slower Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

databricks/megablocks#166 · 1 comment · Opened Mar 3, 2025
bump version for PyTorch 2.6 Open

Difficulty 4/5 3-5 days Newbie friendliness 32/100

databricks/megablocks#165 · 1 reaction · Opened Feb 26, 2025
[rank5]: TypeError: SortOp.forward() takes from 2 to 3 positional arguments but 5 were given When running moe script Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

databricks/megablocks#164 · Opened Feb 8, 2025
Grouped GEMM execution not possible with HW Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

databricks/megablocks#163 · 2 comments · Opened Jan 3, 2025
CUDA Error When Running Single GPU Experiment Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

databricks/megablocks#161 · Opened Nov 14, 2024
Is current megablocks compatible with distributed optimizer in Megatron-LM? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

databricks/megablocks#160 · 1 comment · Opened Nov 11, 2024
amp_C undefined symbol after installing Megablocks Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

databricks/megablocks#157 · 4 comments · Opened Oct 11, 2024
Do you support the bias of mlp？ Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

databricks/megablocks#156 · 1 comment · Opened Oct 9, 2024
what devices are supported? Open

Difficulty 4/5 3-5 days Newbie friendliness 28/100

databricks/megablocks#155 · 15 comments · Opened Oct 8, 2024
Does it work with torch.compile? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

databricks/megablocks#153 · 2 comments · Opened Sep 30, 2024
[Do not Merge] Add CI/CD Milestones Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

databricks/megablocks#132 · Opened Aug 1, 2024
Routing Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

databricks/megablocks#118 · 1 comment · Opened Jun 27, 2024
Illegal memory access on non-0 cuda devices from `histogram` Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

databricks/megablocks#117 · Opened Jun 25, 2024
Can we change self.blocking in dmoe.py from 128 to 64? Open

Difficulty 1/5 Under an hour Newbie friendliness 45/100

databricks/megablocks#114 · 2 comments · Opened May 24, 2024
_LOAD_BALANCING_LOSS returns empty list sometimes Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

databricks/megablocks#113 · 1 comment · Opened May 22, 2024
Bad throughput with GLU Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

databricks/megablocks#110 · 1 comment · Opened May 17, 2024
1-expert worse than dense model Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

databricks/megablocks#107 · 1 comment · Opened May 8, 2024
OSError: Stale file handle with dMoE Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

databricks/megablocks#106 · 3 comments · 2 reactions · Opened Apr 24, 2024
Add a fine-tune script for JetMoE Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

databricks/megablocks#105 · 2 comments · 3 reactions · Opened Apr 17, 2024
ScatterMoE feature Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

databricks/megablocks#104 · 5 comments · Opened Apr 5, 2024
Implement Mixture of Depth and Experts (MoDE) Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

databricks/megablocks#103 · 2 comments · 1 reaction · Opened Apr 5, 2024
Import dmoe model into other training script? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

databricks/megablocks#101 · 3 comments · Opened Apr 2, 2024
SFT Script and Hyperparameters used for DBRX-Instruct Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

databricks/megablocks#99 · 5 comments · Opened Mar 28, 2024
support amd/rocm Open

enhancement help wanted

Difficulty 4/5 3-5 days Newbie friendliness 35/100

databricks/megablocks#97 · 3 comments · Opened Mar 21, 2024
AMP + BF16 failing Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

databricks/megablocks#95 · 7 comments · Opened Jan 28, 2024
the wrong loss func was chosen at evaluation Open

Difficulty 3/5 1-2 days Newbie friendliness 55/100

databricks/megablocks#93 · 2 comments · Opened Jan 18, 2024
Seeking a good multi-node training config Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

databricks/megablocks#92 · 3 comments · 1 reaction · Opened Jan 15, 2024
selective router precision Open

question

Difficulty 5/5 Over a week Newbie friendliness 25/100

databricks/megablocks#91 · 1 comment · Opened Jan 14, 2024
Does this framework support SFT? Open

question

Difficulty 5/5 Over a week Newbie friendliness 15/100

databricks/megablocks#90 · 2 comments · Opened Jan 12, 2024
RuntimeError: Triton Error [CUDA]: invalid argument Open

question

Difficulty 4/5 3-5 days Newbie friendliness 25/100

databricks/megablocks#88 · 15 comments · Opened Jan 10, 2024
different load_balancing_loss with different pipeline_parallel_size Open

question

Difficulty 4/5 3-5 days Newbie friendliness 35/100

databricks/megablocks#85 · 8 comments · Opened Jan 5, 2024
How to integrate to transformers-based mixtral Open

question

Difficulty 5/5 Over a week Newbie friendliness 20/100

databricks/megablocks#84 · 1 comment · Opened Jan 3, 2024
ParallelDroplessMLP initialises self.mlp twice Open

enhancement help wanted

Difficulty 2/5 1-3 hours Newbie friendliness 48/100

databricks/megablocks#83 · 7 comments · 1 reaction · Opened Jan 1, 2024
Why the second matrix of the mlp layer has the same shape of the first one? Open

question

Difficulty 5/5 Over a week Newbie friendliness 25/100

databricks/megablocks#81 · 1 comment · Opened Dec 29, 2023
[BUG] Optimizer Weights Not Reloaded When Training with bf16 Pretrained Weights Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

databricks/megablocks#80 · 1 comment · Opened Dec 26, 2023
Script for Full Fine-Tuning of Mixtral Open

question

Difficulty 5/5 Over a week Newbie friendliness 25/100

databricks/megablocks#68 · 1 comment · 4 reactions · Opened Dec 20, 2023
Docker issues with PyPI installation Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

databricks/megablocks#67 · 3 comments · 1 reaction · Opened Dec 20, 2023
Comparison against top-2 routing? Open

question

Difficulty 5/5 Over a week Newbie friendliness 25/100

databricks/megablocks#49 · 4 comments · Opened Dec 8, 2023

databricks/megablocks

Get new issues in your inbox