aws-samples / aws-samples/foundation-model-benchmarking-tool

Add benchmarking for training models

Open
#270 0 comments 0 reactions 1 assignee Claimed by @madhurprash View on GitHub
Dominant language
Jupyter Notebook
Stars
256
Forks
44
PR merge metrics
No merged PRs in 30d

Description

There are several use cases that require benchmarking, which scopes out of just inferences. This issue is created for integrating training benchmarking into FMBench.

We will experiment with training a llama3-8b on trn1.32xlarge using Hugging Face's Optimum Neuron library. Optimum Neuron is the interface between the Transformers library and AWS Accelerators. It provides a set of tools enabling easy model loading, training and inference on single- and multi-Accelerator settings for different downstream tasks. The list of officially validated models and tasks is available here. Users can try other models and tasks with only few changes.

Link: https://huggingface.co/docs/optimum-neuron/en/training_tutorials/finetune_llm

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.