NVIDIA / NVIDIA/Megatron-LM

Tutorial for running a training on AirGapped cluster with singularity

Open
#1,851 1 comment 2 reactions 0 assignees View on GitHub
community-request enhancement waiting-on-customer
Dominant language
Python
Stars
17.9k
Forks
4.5k
Avg merge
4d 6h
Merged PRs (30d)
271

Description

**Is your feature request related to a problem? Please describe.**
Many national clusters are based on slurm and do not support docker containers, but only singularity. This makes more challenging preparing the environment and then running the experiments (need to download the models and data in advance).

**Describe the solution you'd like**
I am attaching a guide I prepared to support a reasearch group working with Megatron-LM.

**Additional context**
I am an SA at NVIDIA supporting HER so this is a very common problem among people I work with.

[mt-slm-to-publish.md](https://github.com/user-attachments/files/22795262/mt-slm-to-publish.md)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.