aws / aws/sagemaker-training-toolkit

Custom_Overrides

Open
#112 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
530
Forks
140
Avg merge
1h 12m
Merged PRs (30d)
2

Description

**Describe the feature you'd like**
I would like to have more customizability within the SageMaker Training Toolkit. With the ability of a new parameter: custom_override (a dict that can be used to override different commands within the toolkit).

**How would this feature be used? Please describe.**
This would enable users to have more control over their training jobs. For instance, you could use a custom launcher torch.distributed.launch, horovodrun, deepspeed, etc. Moreover, users would be able to add in overrides for special hyperparameters, or other features within custom or prebuilt docker images.

**Describe alternatives you've considered**
You could technically create a workaround where you scrape the hyperparamaters for a CUSTOM_OVERRIDE flag, remove it from the hyperparameters provided from the sdk, then modify all the commands appropriately. This would definitely be a "hackish" solution and would be harder to communicate to users.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.