Azure / Azure/azureml-examples
Can not set instance count when using Batch Endpoint. (From job)
- Dominant language
- Jupyter Notebook
- Stars
- 2k
- Forks
- 1.7k
- Avg merge
- 18h 18m
- Merged PRs (30d)
- 2
Description
### Describe your suggestion
https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-batch-pipeline-from-job?view=azureml-api-2&tabs=cli
I have a Pipeline job with 2 steps. The first step is configured with 2 instance counts. However, when I publish this job to Batch Endpoint, the instance count becomes 1, and there is no other way to set the instance count when calling batch endpoint.
When published to Batch Endpoint, the instance_count and process_count_per_node not works, but the run_invocation_timeout in environment variables is works.
`
"Jobs": {
"flow_step": {
"type": "parallel",
"resources": {
"instance_count": 2
},
"error_threshold": -1,
"environment_variables": {
"AZUREML_PARAMETER_aml_run_invocation_timeout": "13800",
"AZUREML_PARAMETER_aml_run_max_try": "3",
"AZUREML_PARAMETER_aml_process_count_per_node": "8",
"AZUREML_PARAMETER_aml_max_concurrency_per_instance": "8",
"AZUREML_PARAMETER_aml_error_threshold": "-1"
},
"max_concurrency_per_instance": 8,
`
### Additional details
My step is a PromptFlow step and is a parallel job step.
Also, in PromptFlow, the instance count settings are after the flow_component.
https://microsoft.github.io/promptflow/cloud/azureai/use-flow-in-azure-ml-pipeline.html#component-ports-and-run-settings
Contributor guide
Research direction
Start with the Azure Machine Learning batch pipeline and PromptFlow documentation linked in the issue, then reproduce a two-step pipeline whose parallel PromptFlow step uses instance_count 2. Compare the published Batch Endpoint behavior with the original job, including instance_count and process_count_per_node; the issue is done when those settings can be configured and take effect when invoking the endpoint.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100