aws / aws/amazon-sagemaker-examples

How do you use the custom generator to train the TensorFlow model on PageMaker?

Open
#4,574 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

I am trying to train a TensorFlow custom classification model using a custom data generator on SageMaker Studio. However, the notebook did not utilize the GPU to train the model.

My instance type:
![image](https://github.com/aws/amazon-sagemaker-examples/assets/46018083/2a18f0fb-8d9f-4013-b5e3-57f591a57314)

I have stored the images in a zip file on an S3 bucket and unzipped them during training, using the temporary path in the custom generator for training.

I want to add data augmentation while training the model.

The example below loads the images in numpy format and passes them directly to the training model. I want to load the images from the S3 bucket in zip format and train with a custom generator and augmentation.

[Link to the example](https://sagemaker-examples.readthedocs.io/en/latest/sagemaker-experiments/sagemaker_job_tracking/tensorflow_script_mode_training_job.html)

Contributor guide

Open the contributing guide

Research direction

Start with the linked TensorFlow script-mode training example and compare its data-loading flow with the requested S3 zip, custom generator, and augmentation setup. Done would be a clear example or documented guidance showing how this setup trains the model on SageMaker Studio and uses the GPU.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, jupyter-notebook, tensorflow
Domain
cloud, documentation, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.