Azure / Azure/azure-cli

az ml batch-endpoint invoke wants an AccountKey or SasToken for registered Datastores for output

Open
#25,846 24 comments 0 reactions 0 assignees View on GitHub
act-codegen-extensibility-squad Auto-Assign customer-reported Machine Learning Service Attention
Dominant language
Python
Stars
4.6k
Forks
3.5k
Avg merge
3d 2h
Merged PRs (30d)
60

Description

From here https://learn.microsoft.com/en-us/cli/azure/ml/batch-endpoint?view=azure-cli-latest#az-ml-batch-endpoint-invoke

This command works as the output is to the registered internal Azure Machine Learning Default Datastore...

`az ml batch-endpoint invoke --name mybatchendpoint --input https://azuremlexampledata.blob.core.windows.net/data/mnist/sample --input-type uri_folder --output-path azureml://datastores/workspaceblobstore/paths/mybatchendpoint --set output_file_name=predictions.csv --query name -o tsv --resource-group myrg --workspace-name myamlworkspace`

When I want my output to go to another Storage Account location - I have pre-registered it in AML Studio as follows...(under Data > Datastores)

Datastore name: `data_mystorageaccount`
Datastore type: Azure Blob Storage
Subscription ID: {redactedmyAzureSub}
Storage account: `mystorageaccount`
Blob container: `output`
Save credentials with the datastore for data access - `enabled`
Authentication type: `Account key`
Account key: `{the account key from the storage account mystorageaccount}`
Clicked Create

From looking in the Studio it can browse to that output location - see image - it can see a file already called `ServiceTags_Public_20230306.json` in the right hand pane...

![invoke](https://user-images.githubusercontent.com/107028703/225648464-351fcb98-165b-4280-baef-93c387e7503e.PNG)

This proves connection to the Storage Account via the AccountKey as successful.

When I run the following command...

`az ml batch-endpoint invoke --name mybatchendpoint --input https://azuremlexampledata.blob.core.windows.net/data/mnist/sample --input-type uri_folder --output-path azureml://datastores/data_mystorageaccount/paths/mybatchendpoint --set output_file_name=predictions.csv --query name -o tsv --resource-group myrg --workspace-name myamlworkspace`

...it fails with the following output error `Missing AccountKey or SasToken`

```
ERROR: {
"error": {
"code": "UserError",
"severity": null,
"message": "Missing AccountKey or SasToken",
"messageFormat": null,
"messageParameters": null,
"referenceCode": null,
"detailsUri": null,
"target": null,
"details": [],
"innerError": null,
"debugInfo": null,
"additionalInfo": null
},
"correlation": {
"operation": "{redacted}",
"request": "{redacted}"
},
"environment": "{redacted}",
"location": "{redacted}",
"time": "2023-03-16T13:41:12.1918203+00:00",
"componentName": "managementfrontend"
}
```

It states on the `Datastores` screen in AML Studio the following...

`Datastores securely connect to a storage service on Azure by storing connection information. With datastores, you no longer need to provide credential information in your scripts to access your data`

...so I'm not sure why this invoke command fails? Why does it need a AccountKey or SasToken passed to it as part of the az ml batch-endpoint invoke command?

Even if I did need to pass a AccountKey or SasToken as part of this command, is it supported here? There are no examples listed in the documentation for output types

---
#### Document Details

⚠ *Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.*

* ID: 5233a95c-48d7-d49b-3e21-b42c94a31290
* Version Independent ID: 60a98b3a-a7b9-ae58-ab5e-4b1d086a7dd0
* Content: [az ml batch-endpoint](https://learn.microsoft.com/en-us/cli/azure/ml/batch-endpoint?view=azure-cli-latest)
* Content Source: [latest/docs-ref-autogen/ml/batch-endpoint.yml](https://github.com/MicrosoftDocs/azure-docs-cli/blob/main/latest/docs-ref-autogen/ml/batch-endpoint.yml)
* GitHub Login: @rloutlaw
* Microsoft Alias: **routlaw**

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.