Azure / Azure/azure-sdk-for-python

Bug: Optional `command` Component Input Not Working—Fails When Argument Not Passed or Empty (AzureML SDK v2)

Open
#41,834 3 comments 1 reaction 1 assignee Assigned to @Copilot View on GitHub
Client customer-reported Machine Learning needs-team-attention question Service Attention
Dominant language
Python
Stars
5.6k
Forks
3.4k
Avg merge
1d 21h
Merged PRs (30d)
193

Description

- **Package Name**: `azure-ai-ml`
- **Package Version**: 1.27.1
- **Operating System**: Ubuntu 22.04.5 LTS
- **Python Version**: Python 3.12.11

**Describe the bug**
When defining a `command` component in AzureML SDK v2, a string input marked as `optional=True` with a default value *(or not)* does not behave as expected.
If I either omit the input or pass an empty string (""), I receive a `Request is invalid and/or missing fields. The Value field is required.` error, even though the documentation [states this is supported](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-manage-inputs-outputs-pipeline?view=azureml-api-2&tabs=cli#:~:text=Learning%20examples%20repository.-,Define%20optional%20inputs,-By%20default%2C%20all).

**To Reproduce**
Steps to reproduce the behavior:
1. Define a component input as optional:

```python
from azure.ai.ml.entities import Input

inputs = {
"run_date" : Input(type=STRING),
"tags" : Input(type=STRING, optional=True, default="yolo")
}
```
2. Use the following command in the component:

```python
arguments = [
"--run_date ${{inputs.run_date}}",
"$[[--tags ${{inputs.tags}}]]"
]
```

3. Submit a job without specifying `"tags"` in `inputs_run`, or set `"tags": ""`.
```python
python_step_component.compute = COMPUTE_TARGET_RUN
inputs_run = {
"run_date": "2023-01-31",
}
# Submit the job
job = ml_client.jobs.create_or_update(
python_step_component(
**inputs_run
),
experiment_name="test-optional-args",

)
```

4. Error occurs:
```swift
HttpResponseError: (UserError) Request is invalid and/or missing fields.
(RequestInvalid) The Value field is required.
```

Click for full error

```swift

HttpResponseError Traceback (most recent call last)
Cell In[6], line 13
5 inputs_run = {
6 "run_date": "***redacted****",
7 "use_case": "***redacted****",
8 "kedro_pipeline_to_run": "***redacted****",
9 # "tags": "aa" # Optional, default is ''
10 }
12 # 4. Submit the component as a standalone job
---> 13 job = ml_client.jobs.create_or_update(
14 python_step_component(**inputs_run),
15 experiment_name="nptb_bpyn_mt"
16 )
18 print(f"Submitted job name: {job.name}")

File /anaconda/envs/nptb_bpyn/lib/python3.12/site-packages/azure/core/tracing/decorator.py:138, in distributed_trace..decorator..wrapper_use_tracer(*args, **kwargs)
136 for key, value in span_attributes.items():
137 span.add_attribute(key, value) # type: ignore
--> 138 return func(*args, **kwargs)
139 else:
140 # Native path
141 config = {}

File /anaconda/envs/nptb_bpyn/lib/python3.12/site-packages/azure/ai/ml/_telemetry/activity.py:371, in monitor_with_telemetry_mixin..monitor..wrapper(*args, **kwargs)
369 dimensions = {**parameter_dimensions, **(custom_dimensions or {})}
370 with log_activity(logger, activity_name or f.__name__, activity_type, dimensions) as activityLogger:
--> 371 return_value = f(*args, **kwargs)
372 if not parameter_dimensions:
373 # collect from return if no dimensions from parameter
374 activityLogger.activity_info.update(_collect_from_return_value(return_value))

File /anaconda/envs/nptb_bpyn/lib/python3.12/site-packages/azure/ai/ml/operations/_job_operations.py:713, in JobOperations.create_or_update(self, job, description, compute, tags, experiment_name, skip_validation, **kwargs)
707 if (rest_job_resource.properties.job_type == RestJobType.PIPELINE) or (
708 hasattr(rest_job_resource.properties, "identity")
709 and (isinstance(rest_job_resource.properties.identity, UserIdentity))
710 ):
711 self._set_headers_with_user_aml_token(kwargs)
--> 713 result = self._create_or_update_with_different_version_api(rest_job_resource=rest_job_resource, **kwargs)
715 if is_local_run(result):
716 ws_base_url = self._all_operations.all_operations[
717 AzureMLResourceType.WORKSPACE
718 ]._operation._client._base_url

File /anaconda/envs/nptb_bpyn/lib/python3.12/site-packages/azure/ai/ml/operations/_job_operations.py:761, in JobOperations._create_or_update_with_different_version_api(self, rest_job_resource, **kwargs)
758 if rest_job_resource.properties.job_type == RestJobType.COMMAND:
759 service_client_operation = self.service_client_01_2025_preview.jobs
--> 761 result = service_client_operation.create_or_update(
762 id=rest_job_resource.name,
763 resource_group_name=self._operation_scope.resource_group_name,
764 workspace_name=self._workspace_name,
765 body=rest_job_resource,
766 **kwargs,
767 )
769 return result

File /anaconda/envs/nptb_bpyn/lib/python3.12/site-packages/azure/core/tracing/decorator.py:138, in distributed_trace..decorator..wrapper_use_tracer(*args, **kwargs)
136 for key, value in span_attributes.items():
137 span.add_attribute(key, value) # type: ignore
--> 138 return func(*args, **kwargs)
139 else:
140 # Native path
141 config = {}

File /anaconda/envs/nptb_bpyn/lib/python3.12/site-packages/azure/ai/ml/_restclient/v2025_01_01_preview/operations/_jobs_operations.py:632, in JobsOperations.create_or_update(self, resource_group_name, workspace_name, id, body, **kwargs)
630 map_error(status_code=response.status_code, response=response, error_map=error_map)
631 error = self._deserialize.failsafe_deserialize(_models.ErrorResponse, pipeline_response)
--> 632 raise HttpResponseError(response=response, model=error, error_format=ARMErrorFormat)
634 if response.status_code == 200:
635 deserialized = self._deserialize('JobBase', pipeline_response)

HttpResponseError: (UserError) Request is invalid and/or missing fields.
Code: UserError
Message: Request is invalid and/or missing fields.
Exception Details: (RequestInvalid) The Value field is required.
Code: RequestInvalid
Message: The Value field is required.
Additional Information:Type: ComponentName
Info: {
"value": "managementfrontend"
}Type: Correlation
Info: {
"value": {
"operation": "dfc7960391fcc82d77649e2257ea3d0c",
"request": "a7bae414cded0d35"
}
}Type: Environment
Info: {
"value": "eastus2"
}Type: Location
Info: {
"value": "eastus2"
}Type: Time
Info: {
"value": "2025-06-30T19:57:29.6789526+00:00"
}Type: InnerError
Info: {
"value": {
"code": "RequestInvalid",
"innerError": null
}
}Type: MessageFormat
Info: {
"value": "Request is invalid and/or missing fields."
}
```

5. If `"tags": "a"` (non-empty), the job works.

**Expected behavior**
I expect that omitting the `"tags"` input, or passing an empty string, would work without error, as `"tags"` is defined as optional with a default.

This matches the official [documentation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-manage-inputs-outputs-pipeline?view=azureml-api-2&tabs=cli#define-optional-inputs).

**Screenshots**
*N/A (errors shown in stack trace).*

**Additional context**
This issue blocks workflows that expect empty or missing values to use the default.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.