microsoft / microsoft/AzureTRE

Enable use of multiple deployment identities so can switch between GitHub Actions & local Dev Container builds

Open
#4,138 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature has workaround
Dominant language
Python
Stars
235
Forks
192
Avg merge
1d 23h
Merged PRs (30d)
13

Description

Summary

Deploying using GitHub Actions then switching and deploying (to the same TRE instance) using a DevContainer errors during the DevContainer deployment due to KeyVault permissions issue.

Problem occurs because the deployment methods use different user accounts (service principal vs user account) & Key Vault RBAC permissions are set up for one during the first deployment, and terraform errors when trying to read the Key Vault during the plan stage of the second deployment.

Steps to recreate

1. Deploy new TRE instance using GitHub Actions

Key Vault is set up with 'Key Vault Secrets Administrator' role applied to the CICD service principal account

2. Redeploy to same TRE instance using DevContainer, error occurs:

Terraform cannot read KV secrets during the plan stage as there is no 'Key Vault Secrets Administrator' RBAC role set up for the current user.

╷
│ Error: making Read request on Azure KeyVault Secret porter-db-connection-string: keyvault.BaseClient#GetSecret: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="Forbidden" Message="Caller is not authorized to perform action on resource.\r\nIf role assignments, deny assignments or role definitions were changed recently, please observe propagation time.\r\nCaller: appid=04b07795-8ddb-461a-bbee-xxxxxxxxx;oid=8c1b696d-3423-415e-b4ba-xxxxxxxxx;iss=https://sts.windows.net/bcc87841-98cf-40e6-a2a0-aaaaaaaaaaa/\r\nAction: 'Microsoft.KeyVault/vaults/secrets/getSecret/action'\r\nResource: '/subscriptions/10420f14-6516-xxxx-xxxx-xxxxxxxxxxxx/resourcegroups/rg-xxxxxxxx/providers/microsoft.keyvault/vaults/kv-xxxxxxxx/secrets/porter-db-connection-string'\r\nAssignment: (not found)\r\nDenyAssignmentId: null\r\nDecisionReason: null \r\nVault: kv-xxxxxxxx;location=uksouth\r\n" InnerError={"code":"ForbiddenByRbac"}
│ 
│   with azurerm_key_vault_secret.cosmos_mongo_connstr,
│   on cosmos_mongo.tf line 94, in resource "azurerm_key_vault_secret" "cosmos_mongo_connstr":
│   94: resource "azurerm_key_vault_secret" "cosmos_mongo_connstr" {
│ 
╵
╷
│ Error: making Read request on Azure KeyVault Secret api-client-id: keyvault.BaseClient#GetSecret: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="Forbidden" Message="Caller is not authorized to perform action on resource.\r\nIf role assignments, deny assignments or role definitions were changed recently, please observe propagation time.\r\nCaller: appid=04b07795-8ddb-xxxx-xxxx-xxxxxxxxx;oid=8c1b696d-3423-xxxx-xxxx-xxxxxxxxxxx;iss=https://sts.windows.net/bcc87841-98cf-40e6-a2a0-xxxxxxxxx/\r\nAction: 'Microsoft.KeyVault/vaults/secrets/getSecret/action'\r\nResource: '/subscriptions/10420f14-xxxx-xxxx-xxxx-xxxxxxxxx/resourcegroups/rg-nwsdetest/providers/microsoft.keyvault/vaults/kv-nwsdetest/secrets/api-client-id'\r\nAssignment: (not found)\r\nDenyAssignmentId: null\r\nDecisionReason: null \r\nVault: kv-xxxxxxxxx;location=uksouth\r\n" InnerError={"code":"ForbiddenByRbac"}
│ 
│   with azurerm_key_vault_secret.api_client_id,
│   on keyvault.tf line 57, in resource "azurerm_key_vault_secret" "api_client_id":
│   57: resource "azurerm_key_vault_secret" "api_client_id" {
│ 
╵

(etc)

Workaround

Manually assign a Key Vault Administrator role to your user account at a scope higher than the Key Vault itself, such as the resource group or the subscription. If you assign to the Key Vault itself, terraform will error when trying to replace the role on the DevContainer deployment:

│ Error: authorization.RoleAssignmentsClient#Create: Failure responding to request: StatusCode=409 -- Original Error: autorest/azure: Service returned an error. Status=409 Code="RoleAssignmentExists" Message="The role assignment already exists."
│ 
│   with azurerm_role_assignment.keyvault_deployer_role,
│   on keyvault.tf line 14, in resource "azurerm_role_assignment" "keyvault_deployer_role":
│   14: resource "azurerm_role_assignment" "keyvault_deployer_role" {
│ 

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the deployment identity and role-assignment behavior in keyvault.tf, then inspect the Terraform resources named in keyvault.tf and cosmos_mongo.tf. Reproduce a deployment through GitHub Actions followed by the DevContainer deployment against the same TRE instance. Done means both deployment methods can plan and deploy without Key Vault 403 errors or RoleAssignmentExists conflicts.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, github-actions, terraform
Domain
authorization, cloud, devops, infrastructure, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.