AI-Hypercomputer / AI-Hypercomputer/maxtext

Make maxtext_xpk_runner support other trainers

Open
#1,552 0 comments 0 reactions 2 assignees Claimed by @suexu1025 View on GitHub
feature request
Dominant language
Python
Stars
2.4k
Forks
607
Avg merge
2d 19h
Merged PRs (30d)
158

Description

At the moment, the command is [hard coded](https://github.com/AI-Hypercomputer/maxtext/blob/1e4d513ad70dd4074d975a9f7936295008d4b900/benchmarks/maxtext_xpk_runner.py#L412C20-L412C33) to execute the `MaxText.train` module. If the module to execute could be configured in the WorkloadConfig, that would allow for additional trainers being developed to make use of the benchmark runner.

I am running into this while wanting to test a new trainer `MaxText.elastic_train` in PR #1310 but there are also other trainers being developed and merged and I think configuring `maxtext_xpk_runner` will be useful moving forward.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.