Azure / Azure/Azure-Functions

Azure Identity Managed Identity Credential Issue

Open
#2,374 9 comments 0 reactions 1 assignee Claimed by @vijaykumar911 View on GitHub
bug Needs: Triage (functions)
Dominant language
PowerShell
Stars
1.1k
Forks
215
Avg merge
4h 2m
Merged PRs (30d)
1

Description

.
**Library name and version**
Azure.Identity 1.8.2, Azure.Core.DigitalTwins 1.4.0

**Describe the bug**
I am getting an issue when updating the Azure Digital Twins twin values using Managed Credential for Azure Function App.
Already raise this with Azure SDK Team: https://github.com/Azure/azure-sdk-for-net/issues/36210

When I restart the Function app, it works for about ~5-10 minutes but after I see a noticeable lag in the values being updated in Azure Digital Twins due to the Azure Function still 'Executing'. Please note: there are multiple events arriving to Azure Functions app all requiring to update something in Azure Digital Twins. After a while, I see these errors:

`2023-05-12T16:14:53Z [Information] [16:14:53:486][Informational] ManagedIdentityCredential.GetToken was unable to retrieve an access token. Scopes: [ https://digitaltwins.azure.net/.default ] ParentRequestId: 9XXXXXXXXXXXX9 Exception: Azure.Identity.AuthenticationFailedException (0x80131500): ManagedIdentityCredential authentication failed: Retry failed after 4 tries. (The operation was canceled.) (The operation was canceled.) (The operation was canceled.) (The operation was canceled.)
---> System.AggregateException (0x80131500): Retry failed after 4 tries. (The operation was canceled.) (The operation was canceled.) (The operation was canceled.) (The operation was canceled.)
---> System.Threading.Tasks.TaskCanceledException (0x8013153b): The operation was canceled.
---> System.IO.IOException (0x80131620): Unable to read data from the transport connection: Operation canceled.
---> System.Net.Sockets.SocketException (0x80004005): Operation canceled
2023-05-12T16:14:54Z [Information] [16:14:53:608][Informational] Request [eXXXXXXXXXXX] exception System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
---> System.Net.Sockets.SocketException (125): Operation canceled
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.Http.HttpConnection.FillAsync()
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at Azure.Core.Pipeline.HttpClientTransport.ProcessAsync(HttpMessage message)
at Azure.Core.Pipeline.RequestActivityPolicy.ProcessNextAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean isAsync)
at Azure.Core.Pipeline.RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean isAsync)
at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
at Azure.Core.Pipeline.LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)`

Expected behavior
Process the events within milliseconds in real-time

Actual behavior
Severe lag after 5-10 minutes of restarting Function App followed by the error shown above.

Reproduction Steps
For Azure Digital Twins client connection I am using this code (I know this is working because in the first 5 minutes it works as expected). System ManagedIdentity is enabled for Function App and added to ADT Data Owner role - I have seen values being updated in ADT.

var cred = new DefaultAzureCredential();
var client = new DigitalTwinsClient(new Uri(adtInstanceUrl), cred);

this steps below seems to be taking too long (and failing) after a while:
await client.UpdateDigitalTwinAsync(deviceId, updateTwinData);

Environment
Azure Function App Linux 3.19.2.0 (updated now to 4.17.3.3) Consumption Plan (Y1:0)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.