aws / aws/sagemaker-python-sdk
ModelTrainer API cannot configure the right Session
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 1.3k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 35
Description
**Describe the bug**
I'm trying to train a model using the `sagemaker.modules.train.ModelTrainer` API. However, it keeps trying to validate the SageMaker session using Pydantic, only never to accept any possible input. It spits out the Validation Error you see in the screenshot attached below.
**To reproduce**
1. Write a ModelTrainer compatible script
2. Write the following code:
```python
model_trainer = ModelTrainer(
training_image=image_uri, compute=compute, source_code=source_code,
hyperparameters=hyperparameters, environment=env,
base_job_name=job_prefix,
stopping_condition=StoppingCondition(max_runtime_in_seconds=90000),
checkpoint_config=CheckpointConfig(s3_uri=f"{checkpoint_s3_path}/{job_prefix}"),
)
model_trainer.fit(...)
```
**Expected behavior**
Training to start
**Screenshots or logs**
**System information**
A description of your system. Please provide:
- **SageMaker Python SDK version**: 2.248.0
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**: N/A
- **Framework version**: N/A
- **Python version**: 3.12
- **CPU or GPU**: GPU
- **Custom Docker image (Y/N)**: N
**Additional context**
Tried to downgrade to other SageMaker SDK versions, but couldn't get to a working one.
Contributor guide
Research direction
Start with the ModelTrainer API and reproduce the supplied fit(...) example using SageMaker Python SDK 2.248.0 and Python 3.12. Trace the Pydantic validation of the SageMaker session and determine what valid session input is accepted; done means ModelTrainer.fit(...) starts training without a validation error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- api, cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100