Azure / Azure/azure-functions-host

Unable to run Durable Functions with Azure Storage Emulator v5.8

Open
#3,795 21 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
2k
Forks
482
Avg merge
2d 12h
Merged PRs (30d)
38

Description

I was successfully running Durable Functions for a few weeks but now all of a sudden I am not able to run them and am getting the following output. Not sure what has changed -- any pointers on what to look at would be most appreciated. I have also included a related stacktrace below.

```
Azure Functions Core Tools (2.1.725 Commit hash: 68f448fe6a60e1cade88c2004bf6491
af7e5f1df)
Function Runtime Version: 2.0.12134.0
[10/14/2018 10:17:59 PM] Building host: startup suppressed:False, configuration
suppressed: False
[10/14/2018 10:17:59 PM] Reading host configuration file 'C:\Users\csheets\sourc
e\repos\FunctionApp4\FunctionApp1\bin\Debug\netstandard2.0\host.json'
[10/14/2018 10:17:59 PM] Host configuration file read:
[10/14/2018 10:17:59 PM] {
[10/14/2018 10:17:59 PM] "version": "2.0"
[10/14/2018 10:17:59 PM] }
[10/14/2018 10:18:38 PM] A host error has occurred
[10/14/2018 10:18:38 PM] Microsoft.WindowsAzure.Storage: Server encountered an i
nternal error. Please try again after some time.
Listening on http://0.0.0.0:7071/
Hit CTRL-C to exit...
```

The following is a stacktrace that is being thrown:

```
System.AggregateException
HResult=0x80131500
Message=One or more errors occurred.
Source=System.Private.CoreLib
StackTrace:
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at Microsoft.WindowsAzure.Storage.Blob.BlobWriteStream.b__8_0()
at Microsoft.WindowsAzure.Storage.Core.Util.CommonUtility.RunWithoutSynchronizationContext(Action actionToRun)
at Microsoft.WindowsAzure.Storage.Blob.BlobWriteStream.Dispose(Boolean disposing)
at System.IO.Stream.Close()
at System.IO.StreamWriter.Dispose(Boolean disposing)
at System.IO.TextWriter.Dispose()
at Microsoft.Azure.WebJobs.Script.WebHost.BlobStorageSecretsRepository.d__22.MoveNext() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Security\KeyManagement\BlobStorageSecretsRepository.cs:line 167
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at Microsoft.Azure.WebJobs.Script.WebHost.BlobStorageSecretsRepository.d__18.MoveNext() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Security\KeyManagement\BlobStorageSecretsRepository.cs:line 128
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at Microsoft.Azure.WebJobs.Script.WebHost.SecretManager.d__26`1.MoveNext() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Security\KeyManagement\SecretManager.cs:line 412
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at Microsoft.Azure.WebJobs.Script.WebHost.SecretManager.d__11.MoveNext() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\Security\KeyManagement\SecretManager.cs:line 82
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
at Microsoft.Azure.WebJobs.Script.WebHost.DefaultScriptWebHookProvider.d__6.MoveNext() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\WebHooks\DefaultScriptWebHookProvider.cs:line 72
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
at Microsoft.Azure.WebJobs.Script.WebHost.DefaultScriptWebHookProvider.GetExtensionWebHookRoute(String extensionName) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\WebHooks\DefaultScriptWebHookProvider.cs:line 64
at Microsoft.Azure.WebJobs.Script.WebHost.DefaultScriptWebHookProvider.GetUrl(IExtensionConfigProvider extension) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\WebHooks\DefaultScriptWebHookProvider.cs:line 49
at Microsoft.Azure.WebJobs.Host.Config.ExtensionConfigContext.GetWebhookHandler() in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Config\ExtensionConfigContext.cs:line 55
at Microsoft.Azure.WebJobs.Extensions.DurableTask.DurableTaskExtension.Microsoft.Azure.WebJobs.Host.Config.IExtensionConfigProvider.Initialize(ExtensionConfigContext context)
at Microsoft.Azure.WebJobs.Host.DefaultExtensionRegistryFactory.Create() in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\DefaultExtensionRegistryFactory.cs:line 38
at Microsoft.Azure.WebJobs.WebJobsServiceCollectionExtensions.<>c.b__1_0(IServiceProvider p) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Hosting\WebJobsServiceCollectionExtensions.cs:line 50
at DryIoc.Microsoft.DependencyInjection.DryIocAdapter.<>c__DisplayClass3_0.b__0(IResolverContext r) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\DryIocAdapter.cs:line 156
at DryIoc.Registrator.<>c__DisplayClass27_0.b__0(IResolverContext r) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 4530
at DryIoc.Factory.<>c__DisplayClass26_1.b__2() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6584
at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7825
at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7810
at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6583
at DryIoc.Factory.GetExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6544
at DryIoc.Factory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6611
at DryIoc.DelegateFactory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7719
at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, Object serviceKey, IfUnresolved ifUnresolved, Type requiredServiceType, Request preResolveParent, Object[] args) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 290
at DryIoc.Factory.<>c__DisplayClass26_1.b__2() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6584
at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7825
at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7810
at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6583
at DryIoc.Factory.GetExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6544
at DryIoc.Factory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6611
at DryIoc.Container.ResolveAndCacheDefaultFactoryDelegate(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 210
at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 195
at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetService(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\JobHostServiceProvider.cs:line 96
at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetRequiredService(Type serviceType) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\JobHostServiceProvider.cs:line 79
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
at Microsoft.Azure.WebJobs.WebJobsServiceCollectionExtensions.<>c.b__1_4(IServiceProvider p) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Hosting\WebJobsServiceCollectionExtensions.cs:line 83
at DryIoc.Microsoft.DependencyInjection.DryIocAdapter.<>c__DisplayClass3_0.b__0(IResolverContext r) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\DryIocAdapter.cs:line 156
at DryIoc.Registrator.<>c__DisplayClass27_0.b__0(IResolverContext r) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 4530
at DryIoc.Factory.<>c__DisplayClass26_1.b__2() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6584
at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7825
at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7810
at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6583
at DryIoc.Factory.GetExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6544
at DryIoc.Factory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6611
at DryIoc.DelegateFactory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7719
at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, Object serviceKey, IfUnresolved ifUnresolved, Type requiredServiceType, Request preResolveParent, Object[] args) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 290
at DryIoc.Factory.<>c__DisplayClass26_1.b__2() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6584
at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7825
at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7810
at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6583
at DryIoc.Factory.GetExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6544
at DryIoc.ReflectionFactory.CreateExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7059
at DryIoc.Factory.GetExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6534
at DryIoc.ReflectionFactory.CreateExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7059
at DryIoc.Factory.GetExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6534
at DryIoc.Factory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6611
at DryIoc.Container.ResolveAndCacheDefaultFactoryDelegate(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 210
at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 195
at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetService(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\JobHostServiceProvider.cs:line 96
at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetRequiredService(Type serviceType) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\JobHostServiceProvider.cs:line 79
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
at Microsoft.Azure.WebJobs.Script.ScriptHostBuilderExtensions.<>c.b__2_5(IServiceProvider p) in C:\azure-webjobs-sdk-script\src\WebJobs.Script\ScriptHostBuilderExtensions.cs:line 115
at DryIoc.Microsoft.DependencyInjection.DryIocAdapter.<>c__DisplayClass3_0.b__0(IResolverContext r) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\DryIocAdapter.cs:line 156
at DryIoc.Registrator.<>c__DisplayClass27_0.b__0(IResolverContext r) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 4530
at DryIoc.Factory.<>c__DisplayClass26_1.b__2() in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6584
at DryIoc.Scope.TryGetOrAdd(ImMap`1 items, Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7825
at DryIoc.Scope.GetOrAdd(Int32 id, CreateScopedValue createValue, Int32 disposalOrder) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7810
at DryIoc.Factory.ApplyReuse(Expression serviceExpr, Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6583
at DryIoc.Factory.GetExpressionOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6544
at DryIoc.Factory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 6611
at DryIoc.DelegateFactory.GetDelegateOrDefault(Request request) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 7719
at DryIoc.Container.ResolveAndCacheDefaultFactoryDelegate(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 210
at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs:line 195
at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetService(Type serviceType, IfUnresolved ifUnresolved) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\JobHostServiceProvider.cs:line 96
at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.JobHostServiceProvider.GetRequiredService(Type serviceType) in C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\DependencyInjection\JobHostServiceProvider.cs:line 79
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
at Azure.Functions.Cli.Actions.HostActions.StartHostAction.d__44.MoveNext()

Inner Exception 1:
StorageException: Server encountered an internal error. Please try again after some time.
```

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with Azure Functions Core Tools 2.1.725, runtime 2.0.12134.0, and Azure Storage Emulator v5.8, using the host.json configuration shown. Start at Security/KeyManagement/BlobStorageSecretsRepository.cs line 167 and follow the calls through SecretManager.cs lines 412 and 82; done means identifying the storage incompatibility and confirming the host starts successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, csharp
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.