Azure / Azure/azureml-examples

Can not set instance count when using Batch Endpoint. (From job)

Open
#3,610 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Jupyter Notebook
Stars
2k
Forks
1.7k
Avg merge
18h 18m
Merged PRs (30d)
2

Description

### Describe your suggestion

https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-batch-pipeline-from-job?view=azureml-api-2&tabs=cli

I have a Pipeline job with 2 steps. The first step is configured with 2 instance counts. However, when I publish this job to Batch Endpoint, the instance count becomes 1, and there is no other way to set the instance count when calling batch endpoint.

When published to Batch Endpoint, the instance_count and process_count_per_node not works, but the run_invocation_timeout in environment variables is works.

`
"Jobs": {
"flow_step": {
"type": "parallel",
"resources": {
"instance_count": 2
},
"error_threshold": -1,
"environment_variables": {
"AZUREML_PARAMETER_aml_run_invocation_timeout": "13800",
"AZUREML_PARAMETER_aml_run_max_try": "3",
"AZUREML_PARAMETER_aml_process_count_per_node": "8",
"AZUREML_PARAMETER_aml_max_concurrency_per_instance": "8",
"AZUREML_PARAMETER_aml_error_threshold": "-1"
},
"max_concurrency_per_instance": 8,
`

### Additional details

My step is a PromptFlow step and is a parallel job step.

Also, in PromptFlow, the instance count settings are after the flow_component.
https://microsoft.github.io/promptflow/cloud/azureai/use-flow-in-azure-ml-pipeline.html#component-ports-and-run-settings

Contributor guide

Open the contributing guide

Research direction

Start with the Azure Machine Learning batch pipeline and PromptFlow documentation linked in the issue, then reproduce a two-step pipeline whose parallel PromptFlow step uses instance_count 2. Compare the published Batch Endpoint behavior with the original job, including instance_count and process_count_per_node; the issue is done when those settings can be configured and take effect when invoking the endpoint.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
cloud, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.