Azure / Azure/azure-rest-api-specs
BUG: Create PrivateEndpoint using PrivateLink's alias may fail in concurrent creation scenario
Nobody has claimed this yet.
- Dominant language
- TypeSpec
- Stars
- 3.1k
- Forks
- 6k
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 444
Description
When creating multiple PrivateEndpoints with PrivateLink's alias may fail with the following error message:
```text
PrivateLinkServiceId Invalid private link service id: {0} passed in private link service connection: {1} passed in input is invalid
```
The put request to create the PrivateLink resource:
```text
PUT https://management.azure.com/subscriptions/xxxxxxxxxxxxx/resourceGroups/acctestRG-privatelink-220818165005420368/providers/Microsoft.Network/privateLinkServices/acctestPLS-220818165005420368?api-version=2021-08-01 HTTP/1.1
Host: management.azure.com
User-Agent: Go/go1.19 (amd64-windows) go-autorest/v14.2.1 Azure-SDK-For-Go/v65.0.0 network/2021-08-01 HashiCorp Terraform/1.2.6 (+https://www.terraform.io) Terraform Plugin SDK/2.10.1 terraform-provider-azurerm/dev pid-222c6c49-1b0a-5959-a213-6608f9eb8820
Content-Length: 962
Authorization: Bearer
Content-Type: application/json; charset=utf-8
X-Ms-Authorization-Auxiliary:
X-Ms-Correlation-Request-Id: 0c5863fa-fb25-5b75-eab0-4802ca30531e
Accept-Encoding: gzip
```
The request's body is:
```json
{
"location": "centralus",
"properties": {
"autoApproval": {
"subscriptions": [
"xxxxxxxxxxxxx"
]
},
"enableProxyProtocol": false,
"fqdns": [],
"ipConfigurations": [
{
"name": "primaryIpConfiguration-220818165005420368",
"properties": {
"primary": true,
"privateIPAddress": "",
"privateIPAddressVersion": "IPv4",
"privateIPAllocationMethod": "Dynamic",
"subnet": {
"id": "/subscriptions/xxxxxxxxxxxxx/resourceGroups/acctestRG-privatelink-220818165005420368/providers/Microsoft.Network/virtualNetworks/acctestvnet-220818165005420368/subnets/acctestsnetservice-220818165005420368"
}
}
}
],
"loadBalancerFrontendIpConfigurations": [
{
"id": "/subscriptions/xxxxxxxxxxxxx/resourceGroups/acctestRG-privatelink-220818165005420368/providers/Microsoft.Network/loadBalancers/acctestlb-220818165005420368/frontendIPConfigurations/acctestpip-220818165005420368"
}
],
"visibility": {
"subscriptions": [
"xxxxxxxxxxxxx"
]
}
},
"tags": {}
}
```
The response is:
```text
HTTP/1.1 201 Created
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 2585
Content-Type: application/json; charset=utf-8
Expires: -1
Retry-After: 10
x-ms-request-id: 2da20903-2d40-4b7b-b545-50dfb9602eab
Azure-AsyncOperation: https://management.azure.com/subscriptions/xxxxxxxxxxxxx/providers/Microsoft.Network/locations/centralus/operations/2da20903-2d40-4b7b-b545-50dfb9602eab?api-version=2021-08-01
x-ms-correlation-request-id: 0c5863fa-fb25-5b75-eab0-4802ca30531e
Azure-AsyncNotification: Enabled
x-ms-arm-service-request-id: 4bc3f756-b4da-4b91-95b9-522032985f74
Strict-Transport-Security: max-age=31536000; includeSubDomains
Server: Microsoft-HTTPAPI/2.0
Server: Microsoft-HTTPAPI/2.0
x-ms-ratelimit-remaining-subscription-writes: 1195
x-ms-routing-request-id: SOUTHEASTASIA:20220818T085145Z:33bc72f4-61e9-47bc-854d-ec7d23274951
X-Content-Type-Options: nosniff
Date: Thu, 18 Aug 2022 08:51:44 GMT
{
"name": "acctestPLS-220818165005420368",
"id": "/subscriptions/xxxxxxxxxxxxx/resourceGroups/acctestRG-privatelink-220818165005420368/providers/Microsoft.Network/privateLinkServices/acctestPLS-220818165005420368",
"etag": "W/\"067e8de9-b44f-473a-901d-0ad58301f3cc\"",
"type": "Microsoft.Network/privateLinkServices",
"location": "centralus",
"tags": {},
"properties": {
"provisioningState": "Updating",
"resourceGuid": "e2736d14-8e82-4051-98d0-cbff65b71fae",
"fqdns": [],
"alias": "acctestpls-220818165005420368.b08a765d-0bcb-41bf-a0ec-755d43737dbc.centralus.azure.privatelinkservice",
"visibility": {
"subscriptions": [
"xxxxxxxxxxxxx"
]
},
"autoApproval": {
"subscriptions": [
"xxxxxxxxxxxxx"
]
},
"enableProxyProtocol": false,
"loadBalancerFrontendIpConfigurations": [
{
"id": "/subscriptions/xxxxxxxxxxxxx/resourceGroups/acctestRG-privatelink-220818165005420368/providers/Microsoft.Network/loadBalancers/acctestlb-220818165005420368/frontendIPConfigurations/acctestpip-220818165005420368"
}
],
"ipConfigurations": [
{
"name": "primaryIpConfiguration-220818165005420368",
"id": "/subscriptions/xxxxxxxxxxxxx/resourceGroups/acctestRG-privatelink-220818165005420368/providers/Microsoft.Network/privateLinkServices/acctestPLS-220818165005420368/ipConfigurations/primaryIpConfiguration-220818165005420368",
"etag": "W/\"067e8de9-b44f-473a-901d-0ad58301f3cc\"",
"type": "Microsoft.Network/privateLinkServices/ipConfigurations",
"properties": {
"provisioningState": "Succeeded",
"privateIPAllocationMethod": "Dynamic",
"subnet": {
"id": "/subscriptions/xxxxxxxxxxxxx/resourceGroups/acctestRG-privatelink-220818165005420368/providers/Microsoft.Network/virtualNetworks/acctestvnet-220818165005420368/subnets/acctestsnetservice-220818165005420368"
},
"primary": true,
"privateIPAddressVersion": "IPv4"
}
}
],
"privateEndpointConnections": [],
"networkInterfaces": [
{
"id": "/subscriptions/xxxxxxxxxxxxx/resourceGroups/acctestRG-privatelink-220818165005420368/providers/Microsoft.Network/networkInterfaces/acctestPLS-220818165005420368.nic.fd797a3e-c0fd-4464-a8cc-32a3f1e21992"
}
]
}
}
```
As we can see, the PrivateLink resource `acctestPLS-220818165005420368` was created successfully, it's alias was `acctestpls-220818165005420368.b08a765d-0bcb-41bf-a0ec-755d43737dbc.centralus.azure.privatelinkservice`
Then we sent a request to create a new PrivateEndpoint using this alias:
```text
PUT https://management.azure.com/subscriptions/xxxxxxxxxxxxx/resourceGroups/acctestRG-privatelink-220818165005420368/providers/Microsoft.Network/privateEndpoints/acctest-privatelink-220818165005420368?api-version=2021-08-01 HTTP/1.1
Host: management.azure.com
User-Agent: Go/go1.19 (amd64-windows) go-autorest/v14.2.1 Azure-SDK-For-Go/v65.0.0 network/2021-08-01 HashiCorp Terraform/1.2.6 (+https://www.terraform.io) Terraform Plugin SDK/2.10.1 terraform-provider-azurerm/dev pid-222c6c49-1b0a-5959-a213-6608f9eb8820
Content-Length: 593
Authorization: Bearer
Content-Type: application/json; charset=utf-8
X-Ms-Authorization-Auxiliary:
X-Ms-Correlation-Request-Id: 0c5863fa-fb25-5b75-eab0-4802ca30531e
Accept-Encoding: gzip
```
The body was:
```json
{
"location": "centralus",
"properties": {
"manualPrivateLinkServiceConnections": [
{
"name": "acctestPLS-220818165005420368",
"properties": {
"groupIds": [],
"privateLinkServiceId": "acctestpls-220818165005420368.b08a765d-0bcb-41bf-a0ec-755d43737dbc.centralus.azure.privatelinkservice",
"requestMessage": "test"
}
}
],
"privateLinkServiceConnections": [],
"subnet": {
"id": "/subscriptions/xxxxxxxxxxxxx/resourceGroups/acctestRG-privatelink-220818165005420368/providers/Microsoft.Network/virtualNetworks/acctestvnet-220818165005420368/subnets/acctestsnetendpoint-220818165005420368"
}
},
"tags": {}
}
```
We used the PrivateLink's alias here, and the response was:
```text
HTTP/1.1 400 Bad Request
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 237
Content-Type: application/json; charset=utf-8
Expires: -1
x-ms-request-id: d8c3b7fa-7169-4ce5-b2ae-4ecc7629d94c
x-ms-correlation-request-id: 0c5863fa-fb25-5b75-eab0-4802ca30531e
x-ms-arm-service-request-id: 3edd9ac3-4ceb-42b1-b1d2-92ca978897df
Strict-Transport-Security: max-age=31536000; includeSubDomains
Server: Microsoft-HTTPAPI/2.0
Server: Microsoft-HTTPAPI/2.0
x-ms-ratelimit-remaining-subscription-writes: 1196
x-ms-routing-request-id: SOUTHEASTASIA: 20220818T085206Z:c744abc8-4534-4774-845a-a588a6856e90
X-Content-Type-Options: nosniff
Date: Thu,
18 Aug 2022 08: 52: 06 GMT
{
"error": {
"code": "InvalidPrivateLinkServiceId",
"message": "PrivateLinkServiceId Invalid private link service id: {0} passed in private link service connection: {1} passed in input is invalid",
"details": []
}
}
```
We believe this issue is caused by some cache-related issue on service side because:
1. It only happens when we create multiple PrivateLinks concurrently
2. If we retry the creation of PrivateEndpoint, we'll get a successful creation eventually.
We can reproduce this issue by creationg multiple PrivateLinks concurrently, then create a PrivateEndpoint with PrivateLink's alias. Can anyone help us solve this issue? Thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No repository file or test is identified. Start by reproducing the concurrent PrivateLink creation and subsequent PrivateEndpoint PUT using the request bodies and alias shown here, then compare retry behavior. Done would require determining a concrete repository change or documenting that the failure is entirely service-side; the issue does not define either outcome.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- api, cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100