aws / aws/amazon-sagemaker-feedback
Long wait on "INFO: Invoking remote function inside conda environment: sagemaker-runtime-env" when executing training job
- Dominant language
- No language data
- Stars
- 10
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
### Product Version
- [ ] Amazon SageMaker Studio Classic
- [X] Amazon SageMaker Studio
- [ ] Issue is not related to SageMaker Studio
### Issue Description
The problem occurs when I create a training job using Sagemaker Python SDK.
```
with RemoteExecutor(instance_type="ml.g4dn.2xlarge", dependencies='./timeseries_env.yml', max_parallel_jobs=1, keep_alive_period_in_seconds=30) as executor:
future = executor.submit(training_job, arg1, arg2)
```
After dependencies are installed from a yaml file (content provided below) the job freezes for more than an hour on the following line: INFO: Invoking remote function inside conda environment: sagemaker-runtime-env.
```
name: my_env
channels:
- conda-forge
- defaults
dependencies:
- python=3.10
- pandas
- scikit-learn
- matplotlib
- pip:
- sagemaker
- s3fs
- fsspec
- neuralforecast
- hyperopt
```
### Expected Behavior
The job doesn't freeze for so long on the mentioned line
### Observed Behavior
_No response_
### Product Category
Jobs
### Feedback Category
Customer Support, Reliability and Stability, Startup Time and Latency
### Other Details
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.