microsoft / microsoft/aspire

Service discovery refresh query timeout

Open
#3,479 0 comments 0 reactions 0 assignees View on GitHub
area-service-discovery
Dominant language
C#
Stars
6.3k
Forks
991
Avg merge
2d 15h
Merged PRs (30d)
196

Description

Since the preview4 (ignored thinking of a temporary issue), now with the 8.0.0-preview.5.24163.6 (the version used in our last deployment) we encountered a huge amount of timout from service discovery internal refresh.

```
[08:18:00 ERR] Error resolving endpoints for service 'http://_grpc.registry-service'.
DnsClient.DnsResponseException: Query 28796 => _grpc._tcp.registry-service.default.svc.cluster.local IN SRV on 11.0.0.10:53 timed out or is a transient error.
---> System.OperationCanceledException: The operation was canceled.
at System.Threading.CancellationToken.ThrowOperationCanceledException()
at DnsClient.LookupClient.ResolveQueryAsync(IReadOnlyList`1 servers, DnsQuerySettings settings, DnsMessageHandler handler, DnsRequestMessage request, LookupClientAudit audit, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at DnsClient.LookupClient.ResolveQueryAsync(IReadOnlyList`1 servers, DnsQuerySettings settings, DnsMessageHandler handler, DnsRequestMessage request, LookupClientAudit audit, CancellationToken cancellationToken)
at DnsClient.LookupClient.QueryInternalAsync(DnsQuestion question, DnsQuerySettings queryOptions, IReadOnlyCollection`1 servers, CancellationToken cancellationToken)
at Microsoft.Extensions.ServiceDiscovery.Dns.DnsSrvServiceEndPointResolver.ResolveAsyncCore() in /_/src/Microsoft.Extensions.ServiceDiscovery.Dns/DnsSrvServiceEndPointResolver.cs:line 39
at Microsoft.Extensions.ServiceDiscovery.Dns.DnsServiceEndPointResolverBase.ResolveAsyncInternal() in /_/src/Microsoft.Extensions.ServiceDiscovery.Dns/DnsServiceEndPointResolverBase.cs:line 110
at Microsoft.Extensions.ServiceDiscovery.Dns.DnsServiceEndPointResolverBase.ResolveAsync(ServiceEndPointCollectionSource endPoints, CancellationToken cancellationToken) in /_/src/Microsoft.Extensions.ServiceDiscovery.Dns/DnsServiceEndPointResolverBase.cs:line 84
at Microsoft.Extensions.ServiceDiscovery.ServiceEndPointWatcher.RefreshAsyncInternal() in /_/src/Microsoft.Extensions.ServiceDiscovery/ServiceEndPointWatcher.cs:line 141
at Microsoft.Extensions.ServiceDiscovery.ServiceEndPointWatcher.RefreshAsyncInternal() in /_/src/Microsoft.Extensions.ServiceDiscovery/ServiceEndPointWatcher.cs:line 235
```

The strange thing is that the communication works fine!
Only the last week we got two times the "The endpoint collection contains no endpoints" error message in production, and this was the reason of why this issue was opened.

If can help the environment is an AKS with kubernetes v1.28.3.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.