Azure / Azure/MachineLearningNotebooks
MpiStep throws Estimator deprecation warnings
- Dominant language
- Jupyter Notebook
- Stars
- 4.4k
- Forks
- 2.6k
- PR merge metrics
- No merged PRs in 30d
Description
Python version: 3.8.1 as running on an Azure ML notebook VM / SDK version `azureml-core-1.32.0`
There are [several issues here](https://github.com/Azure/MachineLearningNotebooks/search?q=mpistep&type=issues) relating to lack of notebook examples for use of `MpiStep` to do a distributed training run in a pipeline, as the official documentation claims there should be. So I'm creating an example for internal use. When I create an MpiStep, I see this set of deprecation warnings:
```
'Estimator' is deprecated. Please use 'ScriptRunConfig' from 'azureml.core.script_run_config' with your own defined environment or an Azure ML curated environment.
WARNING:root:'distributed_backend' parameter will be deprecated. Please use 'distributed_training' instead.
WARNING:azureml.core.environment:'enabled' is deprecated. Please use the azureml.core.runconfig.DockerConfiguration object with the 'use_docker' param instead.
```
The step is set up like this. I'm explicitly trying to replace a `ScriptRunConfig` with a pipeline step, and don't see another way to run a `PythonScriptStep` in parallel than by using an MpiStep.
```
script_step = MpiStep(
name='distributed-train-step',
script_name=script,
compute_target=compute_target,
source_directory=project_folder,
node_count=2,
process_count_per_node=1,
arguments=['--train_data', train_data.as_mount(),
'--test_data', test_data.as_mount(),
'--epochs', 100,
'--lr', 0.005],
custom_docker_image=docker_base_image,
)
```
I understand if this isn't the place to log this kind of generic issue relating to the Azure ML SDK but if so, then where is?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.