aws / aws/amazon-sagemaker-examples

GPT-NeoX

Open
#3,981 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

**Describe the use case example you want to see**
Any example of distributed SMP training with Tensor Parallel for GPT-NeoX. There was mention of benchmarks w/ this architecture back in December 2022. https://aws.amazon.com/blogs/machine-learning/new-performance-improvements-in-amazon-sagemaker-model-parallel-library/

A clear and concise description of the functionality you want.

**How would this example be used? Please describe.**
A clear and concise description of the use case for this example.

**Describe which SageMaker services are involved**

**Describe what other services (other than SageMaker) are involved***

**Describe which dataset could be used. Provide its location in s3://sagemaker-sample-files or another source.**

Contributor guide

Open the contributing guide

Research direction

No file, test, or entry point is named. Start by reviewing the linked AWS blog and the repository's existing SageMaker examples, then clarify the distributed SMP, Tensor Parallel, SageMaker services, other services, and dataset requirements. Done would be a documented GPT-NeoX training example that covers those requirements.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, jupyter-notebook
Domain
cloud, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.