[machinelearningservices] Azure ML workspace stuck in soft-deleted state: create blocked, permanently-delete cannot find workspace

Source: https://github.com/Azure/azure-cli/issues/33756 (by @vairakkumaarsvs)
Affected extension: `machinelearningservices` (`src/machinelearningservices/`)

---

<<>>
# Issue #33756 (by @vairakkumaarsvs)
## Title
Azure ML workspace stuck in soft-deleted state: create blocked, permanently-delete cannot find workspace

## Body
### Describe the bug

Azure CLI Version:
2.88.0

ML Extension:
2.44.1

Subscription:

Resource Group:
stmlwus2ws01-rg

Workspace:
stmlwus2ws01

Problem:

I deleted an Azure ML workspace.

When I attempt to recreate the workspace, deployment fails with:

"Soft-deleted workspace exists. Please purge or recover it."

However, attempting to permanently delete the workspace fails because the workspace cannot be found:

az ml workspace delete \
--name myws \
--resource-group my-rg \
--subscription \
--permanently-delete \
--yes

Error:

(UserError) Unable to find workspace:
/subscriptions/.../resourcegroups/my-rg/providers/Microsoft.MachineLearningServices/workspaces/myws

This creates a deadlock:

- Workspace creation fails because a soft-deleted workspace exists.
- Permanent delete fails because the workspace cannot be found.
- No CLI command appears available to enumerate or purge the soft-deleted workspace.

Expected behavior:

Either:
1. `az ml workspace delete --permanently-delete` should purge the soft-deleted workspace, or
2. The CLI should provide a supported command to list and purge soft-deleted workspaces.

Actual behavior:

Workspace name remains blocked but cannot be purged.

### Related command

az ml workspace delete \
--name myws \
--resource-group my-rg \
--subscription \
--permanently-delete \
--yes

### Errors

(UserError) Unable to find workspace:
Code: UserError
Message: Unable to find workspace:

### Issue script & Debug output

> cli: Received HttpResponseError: Traceback (most recent call last):
File "/home/myubuntuuser/.azure/cliextensions/ml/azext_mlv2/manual/custom/workspace.py", line 278, in ml_workspace_delete
del_result = ml_client.workspaces.begin_delete(
name=name, delete_dependent_resources=all_resources, permanently_delete=permanently_delete
)
File "/home/myubuntuuser/.azure/cliextensions/ml/azure/ai/ml/_telemetry/activity.py", line 288, in wrapper
return f(*args, **kwargs)
File "/opt/az/lib/python3.14/site-packages/azure/core/tracing/decorator.py", line 119, in wrapper_use_tracer
return func(*args, **kwargs)
File "/home/myubuntuuser/.azure/cliextensions/ml/azure/ai/ml/operations/_workspace_operations.py", line 355, in begin_delete
return super().begin_delete(
~~~~~~~~~~~~~~~~~~~~^
name, delete_dependent_resources=delete_dependent_resources, permanently_delete=permanently_delete, **kwargs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/myubuntuuser/.azure/cliextensions/ml/azure/ai/ml/operations/_workspace_operations_base.py", line 494, in begin_delete
workspace: Any = self.get(name, **kwargs)
~~~~~~~~^^^^^^^^^^^^^^^^
File "/home/myubuntuuser/.azure/cliextensions/ml/azure/ai/ml/_telemetry/activity.py", line 288, in wrapper
return f(*args, **kwargs)
File "/opt/az/lib/python3.14/site-packages/azure/core/tracing/decorator.py", line 119, in wrapper_use_tracer
return func(*args, **kwargs)
File "/home/myubuntuuser/.azure/cliextensions/ml/azure/ai/ml/operations/_workspace_operations.py", line 143, in get
return super().get(workspace_name=name, **kwargs)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/myubuntuuser/.azure/cliextensions/ml/azure/ai/ml/operations/_workspace_operations_base.py", line 91, in get
obj = self._operation.get(resource_group, workspace_name)
File "/opt/az/lib/python3.14/site-packages/azure/core/tracing/decorator.py", line 119, in wrapper_use_tracer
return func(*args, **kwargs)
File "/home/myubuntuuser/.azure/cliextensions/ml/azure/ai/ml/_restclient/v2024_10_01_preview_tsp/operations/_operations.py", line 10422, in get
map_error(status_code=response.status_code, response=response, error_map=error_map)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/az/lib/python3.14/site-packages/azure/core/exceptions.py", line 163, in map_error
raise error
azure.core.exceptions.ResourceNotFoundError: (ResourceNotFound) The Resource 'Microsoft.MachineLearningServices/workspaces/mymlws' under resource group 'mymlws-rg' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix
Code: ResourceNotFound
Message: The Resource 'Microsoft.MachineLearningServices/workspaces/mymlws' under resource group 'mymlws-rg' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix

cli.azure.cli.core.azclierror: Traceback (most recent call last):
File "/opt/az/lib/python3.14/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 677, in execute
raise ex
File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 820, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 789, in _run_job
result = cmd_copy(params)
File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 335, in __call__
return self.handler(*args, **kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/command_operation.py", line 120, in handler
return op(**command_args)
File "/home/myubuntuuser/.azure/cliextensions/ml/azext_mlv2/manual/custom/workspace.py", line 285, in ml_workspace_delete
log_and_raise_error(err, debug)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/home/myubuntuuser/.azure/cliextensions/ml/azext_mlv2/manual/custom/raise_error.py", line 193, in log_and_raise_error
raise cli_error
File "/home/myubuntuuser/.azure/cliextensions/ml/azext_mlv2/manual/custom/workspace.py", line 278, in ml_workspace_delete
del_result = ml_client.workspaces.begin_delete(
name=name, delete_dependent_resources=all_resources, permanently_delete=permanently_delete
)
File "/home/myubuntuuser/.azure/cliextensions/ml/azure/ai/ml/_telemetry/activity.py", line 288, in wrapper
return f(*args, **kwargs)
File "/opt/az/lib/python3.14/site-packages/azure/core/tracing/decorator.py", line 119, in wrapper_use_tracer
return func(*args, **kwargs)
File "/home/myubuntuuser/.azure/cliextensions/ml/azure/ai/ml/operations/_workspace_operations.py", line 355, in begin_delete
return super().begin_delete(
~~~~~~~~~~~~~~~~~~~~^
name, delete_dependent_resources=delete_dependent_resources, permanently_delete=permanently_delete, **kwargs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/myubuntuuser/.azure/cliextensions/ml/azure/ai/ml/operations/_workspace_operations_base.py", line 494, in begin_delete
workspace: Any = self.get(name, **kwargs)
~~~~~~~~^^^^^^^^^^^^^^^^
File "/home/myubuntuuser/.azure/cliextensions/ml/azure/ai/ml/_telemetry/activity.py", line 288, in wrapper
return f(*args, **kwargs)
File "/opt/az/lib/python3.14/site-packages/azure/core/tracing/decorator.py", line 119, in wrapper_use_tracer
return func(*args, **kwargs)
File "/home/myubuntuuser/.azure/cliextensions/ml/azure/ai/ml/operations/_workspace_operations.py", line 143, in get
return super().get(workspace_name=name, **kwargs)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/myubuntuuser/.azure/cliextensions/ml/azure/ai/ml/operations/_workspace_operations_base.py", line 91, in get
obj = self._operation.get(resource_group, workspace_name)
File "/opt/az/lib/python3.14/site-packages/azure/core/tracing/decorator.py", line 119, in wrapper_use_tracer
return func(*args, **kwargs)
File "/home/
... [truncated, original was 10958 chars]

## Comments
### Comment by @yonzhan

Thank you for opening this issue, we will look into it.
<<>>