Azure / Azure/MachineLearningNotebooks

MpiStep throws Estimator deprecation warnings

Open
#1,558 1 comment 0 reactions 0 assignees View on GitHub
ADO Core UI doc-enhancement
Dominant language
Jupyter Notebook
Stars
4.4k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

Python version: 3.8.1 as running on an Azure ML notebook VM / SDK version `azureml-core-1.32.0`

There are [several issues here](https://github.com/Azure/MachineLearningNotebooks/search?q=mpistep&type=issues) relating to lack of notebook examples for use of `MpiStep` to do a distributed training run in a pipeline, as the official documentation claims there should be. So I'm creating an example for internal use. When I create an MpiStep, I see this set of deprecation warnings:

```
'Estimator' is deprecated. Please use 'ScriptRunConfig' from 'azureml.core.script_run_config' with your own defined environment or an Azure ML curated environment.
WARNING:root:'distributed_backend' parameter will be deprecated. Please use 'distributed_training' instead.
WARNING:azureml.core.environment:'enabled' is deprecated. Please use the azureml.core.runconfig.DockerConfiguration object with the 'use_docker' param instead.
```

The step is set up like this. I'm explicitly trying to replace a `ScriptRunConfig` with a pipeline step, and don't see another way to run a `PythonScriptStep` in parallel than by using an MpiStep.

```
script_step = MpiStep(
name='distributed-train-step',
script_name=script,
compute_target=compute_target,
source_directory=project_folder,
node_count=2,
process_count_per_node=1,
arguments=['--train_data', train_data.as_mount(),
'--test_data', test_data.as_mount(),
'--epochs', 100,
'--lr', 0.005],
custom_docker_image=docker_base_image,
)
```

I understand if this isn't the place to log this kind of generic issue relating to the Azure ML SDK but if so, then where is?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.