aws / aws/amazon-sagemaker-examples
GPT-NeoX
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
**Describe the use case example you want to see**
Any example of distributed SMP training with Tensor Parallel for GPT-NeoX. There was mention of benchmarks w/ this architecture back in December 2022. https://aws.amazon.com/blogs/machine-learning/new-performance-improvements-in-amazon-sagemaker-model-parallel-library/
A clear and concise description of the functionality you want.
**How would this example be used? Please describe.**
A clear and concise description of the use case for this example.
**Describe which SageMaker services are involved**
**Describe what other services (other than SageMaker) are involved***
**Describe which dataset could be used. Provide its location in s3://sagemaker-sample-files or another source.**
Contributor guide
Research direction
No file, test, or entry point is named. Start by reviewing the linked AWS blog and the repository's existing SageMaker examples, then clarify the distributed SMP, Tensor Parallel, SageMaker services, other services, and dataset requirements. Done would be a documented GPT-NeoX training example that covers those requirements.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, jupyter-notebook
- Domain
- cloud, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100