Azure / Azure/terraform

Terraform deployment with VNET injection

Open
#41 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
HCL
Stars
925
Forks
930
Avg merge
4d 15h
Merged PRs (30d)
5

Description

Hi
I'm trying to deploy databricks with VNET injection.
My code looks like this:

##############################################################################################################################3
# DATABRICKS WORKSPACE
##############################################################################################################################3

resource "azurerm_databricks_workspace" "dp-dbricks-dev" {
name = var.databricks_workspace_suffix
resource_group_name = var.databricks_depends_on # select databricks resource group
location = var.dbricks_location
sku = var.db_ws_sku

managed_resource_group_name = var.databricks_depends_on

custom_parameters {
no_public_ip = true
public_subnet_name = var.snet_pub_addr_name
private_subnet_name = var.snet_prv_addr_name
virtual_network_id = var.vnet_id
}
depends_on = [var.databricks_depends_on]
}
##############################################################################################################################3
# DATABRICKS NSG AND ASSOCIATION RESOURCES
##############################################################################################################################3

resource "azurerm_network_security_group" "service-dap-nsg" {
name = var.service-dap-nsg-name
location = var.dbricks_location
resource_group_name = var.databricks_depends_on
security_rule {
name = "allowedSubnets"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_range = "*"
source_address_prefixes = ["${var.snet_prv_addr_prefix}","${var.snet_link_addr_prefix}","${var.snet_apps_addr_prefix}"]
destination_address_prefix = "*"
}
}
##############################################################################################################################3
#DATABRICKS NSG ASSOCIATION FOR BOTH PRIVATE AND PUBLIC SUBNET
##############################################################################################################################3
resource "azurerm_subnet_network_security_group_association" "dbricks-nsg-prv" { #for private subnet
subnet_id = var.db_subnet_id
network_security_group_id = azurerm_network_security_group.service-dap-nsg.id
depends_on = [azurerm_network_security_group.service-dap-nsg]
}
resource "azurerm_subnet_network_security_group_association" "dbricks-nsg-pub" { #for public subnet
subnet_id = var.db_pub_subnet_id
network_security_group_id = azurerm_network_security_group.service-dap-nsg.id
depends_on = [azurerm_network_security_group.service-dap-nsg]
}

Both subnets have the databricks associated and delegation added, but it keeps failing with error:
Error: Error updating Route Table Association for Subnet "snet-prv-dap-dev-cc-001" (Virtual Network "vnet-dap-dev-cc-001" / Resource Group "rg-dap-core-dev"): network.SubnetsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="ConflictWithNetworkIntentPolicy" Message="Found conflicts with NetworkIntentPolicy. Details: Subnet or Virtual Network cannot have resources or properties which conflict with network intent policy.\r\nNetwork Security Group: /subscriptions/***/resourceGroups/rg-dap-service-dev-001/providers/Microsoft.Network/networkSecurityGroups/nsg-dbks-dap-dev-cc-001 conflicts with Network Intent Policy: adb-canadacentral-npip-57ca56ef13781757ae293ff6\r\n Network Security Group doesn't have supporting Security Rule for Network Intent Policy Security Rule: Name: databricks-worker-to-databricks-webapp, Id: /subscriptions/***/resourceGroups/rg-dap-core-dev/providers/Microsoft.Network/networkIntentPolicies/adb-canadacentral-npip-57ca56ef13781757ae293ff6/securityRules/databricks-worker-to-databricks-webapp, Access: Allow, Direction: Outbound, Protocol: tcp, SourceAddressPrefix: VirtualNetwork, SourcePortRange: *, DestinationAddressPrefix: AzureDatabricks, DestinationPortRange: 443\r\n ----\r\n Network Security Group doesn't have supporting Security Rule for Network Intent Policy Security Rule: Name: databricks-worker-to-sql, Id: /subscriptions/***/resourceGroups/rg-dap-core-dev/providers/Microsoft.Network/networkIntentPolicies/adb-canadacentral-npip-57ca56ef13781757ae293ff6/securityRules/databricks-worker-to-sql, Access: Allow, Direction: Outbound, Protocol: tcp, SourceAddressPrefix: VirtualNetwork, SourcePortRange: *, DestinationAddressPrefix: Sql, DestinationPortRange: 3306\r\n ----\r\n Network Security Group doesn't have supporting Security Rule for Network Intent Policy Security Rule: Name: databricks-worker-to-storage, Id: /subscriptions/***/resourceGroups/rg-dap-core-dev/providers/Microsoft.Network/networkIntentPolicies/adb-canadacentral-npip-57ca56ef13781757ae293ff6/securityRules/databricks-worker-to-storage, Access: Allow, Direction: Outbound, Protocol: tcp, SourceAddressPrefix: VirtualNetwork, SourcePortRange: *, DestinationAddressPrefix: Storage, DestinationPortRange: 443\r\n ----\r\n Network Security Group doesn't have supporting Security Rule for Network Intent Policy Security Rule: Name: databricks-worker-to-eventhub, Id: /subscriptions/***/resourceGroups/rg-dap-core-dev/providers/Microsoft.Network/networkIntentPolicies/adb-canadacentral-npip-57ca56ef13781757ae293ff6/securityRules/databricks-worker-to-eventhub, Access: Allow, Direction: Outbound, Protocol: tcp, SourceAddressPrefix: VirtualNetwork, SourcePortRange: *, DestinationAddressPrefix: EventHub, DestinationPortRange: 9093\r\n ----\r\n---- ----" Details=[]

Could you please help me with that.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the azurerm_databricks_workspace custom_parameters block and the two subnet network security group associations shown in the issue. Compare the reported NetworkIntentPolicy conflicts with the configured NSG rules. Done means the VNET-injected Databricks deployment completes without the ConflictWithNetworkIntentPolicy error.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, terraform
Domain
cloud, infrastructure, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.