microsoft / microsoft/durabletask-netherite
ScheduleNewOrchestrationInstanceAsync fails when scale controller scaling in
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 239
- Forks
- 37
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 2
Description
I've recently started using Netherite (Microsoft.Azure.Functions.Worker.Extensions.DurableTask.Netherite" Version="1.5.1") and have been seeing a few errors when starting orchestrations.
Orchestrations start fine most of the time, but occasionally I see the following error:
When starting an orchestration, as follows:
var ingestionOrchestrationSpec = new IngestionOrchestrationSpec(etc...);
var orchestrationInstanceId = $"ingestions.{tenant.ProjectId}.{pointcloud.Id}";
await client.ScheduleNewOrchestrationInstanceAsync(nameof(IngestionsCreateAsyncOrchestrator), ingestionOrchestrationSpec, new StartOrchestrationOptions { InstanceId = orchestrationInstanceId });
This exception is thrown:
Exception: Grpc.Core.RpcException: Status(StatusCode="Unknown", Detail="Exception was thrown by handler.")
at Microsoft.DurableTask.Client.Grpc.GrpcDurableTaskClient.ScheduleNewOrchestrationInstanceAsync(TaskName orchestratorName, Object input, StartOrchestrationOptions options, CancellationToken cancellation)
at TrimbleGeospatial.Seurat.ApiApp.Functions.Ingestions_Create.Run(HttpRequestData req, DurableTaskClient client, FunctionContext functionContext, String projectId, String pointcloudId)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with ScheduleNewOrchestrationInstanceAsync and the reported Grpc.Core.RpcException while the scale controller is scaling in. Reproduce the failure using the orchestration call shown, then trace the handler and add coverage demonstrating that scheduling succeeds during scale-in without the exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, csharp
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100