Azure / Azure/azure-rest-api-specs
Elasticsearch deployment automation does not work anymore
- Dominant language
- TypeSpec
- Stars
- 3.1k
- Forks
- 5.9k
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 444
Description
Calling
```
PUT https://management.azure.com/subscriptions/309065ca-a060-4592-8096-b74694126b61/resourceGroups/azure-demo-01-monitoring-rg/providers/Microsoft.Elastic/monitors/test-elasticsearch?api-version=2023-02-01-preview
{"location":"westeurope","properties":{"monitoringStatus":"Enabled","userInfo":{"emailAddress":"THES@softwareag.com"}},"sku":{"name":"ess-consumption-2024_Monthly@TIDgmz7xq9ge3py"},"tags":{}}
```
does not work anymore for a token retrieved locally using `az account get-access-token`. I am always getting the following error message:
```
Status 400
{
"error": {
"code": "BadRequest",
"message": "Cannot proceed with the request as the user is not authorized"
}
}
```
It is still working fine with a token retrieved in Azure Cloud Shell (in the Azure Portal) - for the same user.
Some more details:
I have been using Terraform for setting up my Azure infrastructure including an instance of Managed Elasticsearch for several months, and it has always been working fine until yesterday (08.12.23) morning CET. It failed for the first time yesterday (08.12.23) afternoon, and it has failed ever since.
Please find the Terraform configuration of a repro case in the attachment.
Since Friday afternoon, it would always complain:
```
│ Error: creating Monitor (Subscription: "309065ca-a060-4592-8096-b74694126b61"
│ Resource Group Name: "azure-demo-01-monitoring-rg"
│ Monitor Name: "azure-demo-01-elasticsearch"): performing MonitorsCreate: unexpected status 400 with error: BadRequest: Cannot proceed with the request as the user is not authorized
│
│ with module.monitoring.azurerm_elastic_cloud_elasticsearch.default,
│ on monitoring\monitoring-elasticsearch.tf line 1, in resource "azurerm_elastic_cloud_elasticsearch" "default":
│ 1: resource "azurerm_elastic_cloud_elasticsearch" "default" {
```
Yes, I am the owner of the resource group, and Terraform is working fine for dozens of other resources. Yes, I can log-in to Elastic Cloud with my Microsoft account [THES@softwareag.com](mailto:THES@softwareag.com). Yes, I can create an Elasticsearch instance in the same RG with the same resource configuration with the same user in Azure portal.
I have tried the Terraform script with location westeurope, eastus and southeastasia. Same result.
I have tried creating the resource using Azure CLI:
`az elastic monitor create -n test-elasticsearch -g azure-demo-01-monitoring-rg --user-info "{firstName:Thomas,lastName:Hesse,companyName:'Software AG',emailAddress:THES@softwareag.com}" --sku "{name:ess-consumption-2024_Monthly@TIDgmz7xq9ge3py}"`
Result:
```
(BadRequest) Cannot proceed with the request as the user is not authorized
Code: BadRequest
Message: Cannot proceed with the request as the user is not authorized
```
I have exported an ARM template in the Azure Portal, cf. attachment. I can create the resource using the template in the Azure Portal, but it fails when I try the following:
`az deployment group create --resource-group azure-demo-01-monitoring-rg --template-file ExportedTemplate-azure-demo-01-elasticsearch.json --parameters @ExportedTemplate-azure-demo-01-elasticsearch-parameters.json`
Result:
`{"status":"Failed","error":{"code":"DeploymentFailed","target":"/subscriptions/309065ca-a060-4592-8096-b74694126b61/resourceGroups/azure-demo-01-monitoring-rg/providers/Microsoft.Resources/deployments/ExportedTemplate-azure-demo-01-elasticsearch","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.","details":[{"code":"BadRequest","message":"Cannot proceed with the request as the user is not authorized"}]}}`
I have even tried calling the Azure Management API:
```
PUT https://management.azure.com/subscriptions/309065ca-a060-4592-8096-b74694126b61/resourceGroups/azure-demo-01-monitoring-rg/providers/Microsoft.Elastic/monitors/test-elasticsearch?api-version=2023-02-01-preview
{"location":"westeurope","properties":{"monitoringStatus":"Enabled","userInfo":{"emailAddress":"THES@softwareag.com"}},"sku":{"name":"ess-consumption-2024_Monthly@TIDgmz7xq9ge3py"},"tags":{}}
```
Result:
```
Status 400
{
"error": {
"code": "BadRequest",
"message": "Cannot proceed with the request as the user is not authorized"
}
}
```
I have tried multiple versions of the API.
Yes, I am using a valid token, the API is working fine for other resource types.
Some more results of my analysis:
- Terraform is working fine when I run it in an Azure DevOps pipeline. It only fails when I run it locally.
- `az elastic monitor list` is working fine. Only `az elastic monitor create` fails when I run it locally.
- `az elastic monitor create` is working fine when I run it in a Cloud Shell in Azure Portal, but not on my PC.
- Creating a token in Cloud Shell with `az account get-access-token` and using it in Postman on my PC is working fine, but not with a token created with `az account get-access-token` on my PC. When I compare the tokens, I see a difference in the unique_name claim. It is THES@softwareag.com for the token created in Cloud Shell and thomas.hesse@softwareag.com for the token created locally, cf. attachments.
Maybe this is causing the issue? Please note that my Elastic Cloud username is THES@softwareag.com. Are you taking the Elastic Cloud username from the unique_name claim of the token and not from the `userInfo.emailAddress` of the request body? Or are you forwarding the token to Elastic Cloud? (I hope not…)
Anyhow: How can I log-in to Azure CLI in such a way that Terraform and `az elastic monitor create` are working locally again?
[JWT_Payload_Local.json](https://github.com/hashicorp/terraform-provider-azurerm/files/13625319/JWT_Payload_Local.json)
[JWT_Payload_Cloud_Shell.json](https://github.com/hashicorp/terraform-provider-azurerm/files/13625320/JWT_Payload_Cloud_Shell.json)
Currently I see no way of setting up the resource automatically. Please help!
Best regards,
Thomas
Contributor guide
Research direction
Start with monitoring\monitoring-elasticsearch.tf and the attached JWT payloads; reproduce the local `az elastic monitor create` or ARM PUT and compare it with Cloud Shell and pipeline tokens. Done means explaining or correcting the Microsoft.Elastic authorization difference so local Terraform and CLI creation succeeds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, terraform
- Domain
- api, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100