Increase 4MB of gRPC limited in Azure Durable Function for Java

先從 Java worker 的啟動引數追蹤 grpcMaxMessageLength 到 DurableTaskGrpcClient.scheduleNewOrchestrationInstance 路徑，並將其與 Function Host 執行階段設定以及此處描述的 Python 行為進行比較。使用超過 4 MB 的 POST payload，在 Function Host 4.21.1.20667 中於本機重現；當 Java Durable Function 能接受較大的訊息且不出現 gRPC RESOURCE_EXHAUSTED 錯誤時，即表示完成。

There seems to be a limit only in Azure Durable Function For Java that cannot send payload over 4MB, can you update the limit and align with int.MaxValue in function host runtime? Below are the observation about this issue:

1. Got below error when POST to durable function with a payload over 4MB.

Microsoft.Azure.WebJobs.Host.FunctionInvocationException : Exception while executing function: Functions.StartOrchestration ---> Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcException : Result: Failure

Exception: StatusRuntimeException: RESOURCE_EXHAUSTED: Received message larger than max (5060857 vs. 4194304)
Stack: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.microsoft.azure.functions.worker.broker.JavaMethodInvokeInfo.invoke(JavaMethodInvokeInfo.java:22)
at com.microsoft.azure.functions.worker.broker.EnhancedJavaMethodExecutorImpl.execute(EnhancedJavaMethodExecutorImpl.java:22)
at com.microsoft.azure.functions.worker.chain.FunctionExecutionMiddleware.invoke(FunctionExecutionMiddleware.java:19)
at com.microsoft.azure.functions.worker.chain.InvocationChain.doNext(InvocationChain.java:21)
at com.microsoft.durabletask.azurefunctions.internal.middleware.OrchestrationMiddleware.invoke(OrchestrationMiddleware.java:29)
at com.microsoft.azure.functions.worker.chain.InvocationChain.doNext(InvocationChain.java:21)
at com.microsoft.azure.functions.worker.broker.JavaFunctionBroker.invokeMethod(JavaFunctionBroker.java:125)
at com.microsoft.azure.functions.worker.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:34)
at com.microsoft.azure.functions.worker.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:10)
at com.microsoft.azure.functions.worker.handler.MessageHandler.handle(MessageHandler.java:44)
at com.microsoft.azure.functions.worker.JavaWorkerClient$StreamingMessagePeer.lambda$onNext$0(JavaWorkerClient.java:94)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: Received message larger than max (5060857 vs. 4194304)
at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:271)
at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:252)
at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:165)
at com.microsoft.durabletask.implementation.protobuf.TaskHubSidecarServiceGrpc$TaskHubSidecarServiceBlockingStub.startInstance(TaskHubSidecarServiceGrpc.java:919)
at com.microsoft.durabletask.DurableTaskGrpcClient.scheduleNewOrchestrationInstance(DurableTaskGrpcClient.java:114)
at com.microsoft.durabletask.DurableTaskClient.scheduleNewOrchestrationInstance(DurableTaskClient.java:67)
at com.microsoft.durabletask.DurableTaskClient.scheduleNewOrchestrationInstance(DurableTaskClient.java:52)
at com.singpost.wms.uniqlo.function.SplitXMLFunction.startOrchestration(SplitXMLFunction.java:75)
... 20 more

at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker.InvokeCore(Object[] parameters,FunctionInvocationContext context) at //src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs : 96
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) at //src/WebJobs.Script/Description/FunctionInvokerBase.cs : 82
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Description.FunctionGenerator.Coerce[T](Task1 src) at /_/src/WebJobs.Script/Description/FunctionGenerator.cs : 225 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at async Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker2.InvokeAsync[TReflected,TReturnValue](Object instance,Object[] arguments) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.cs : 52
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeWithTimeoutAsync(IFunctionInvoker invoker,ParameterHelper parameterHelper,CancellationTokenSource timeoutTokenSource,CancellationTokenSource functionCancellationTokenSource,Boolean throwOnTimeout,TimeSpan timerInterval,IFunctionInstance instance) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 581
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstanceEx instance,ParameterHelper parameterHelper,ILogger logger,CancellationTokenSource functionCancellationTokenSource) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 527
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance,FunctionStartedMessage message,FunctionInstanceLogEntry instanceLogEntry,ParameterHelper parameterHelper,ILogger logger,CancellationToken cancellationToken) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 306
End of inner exception
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance,FunctionStartedMessage message,FunctionInstanceLogEntry instanceLogEntry,ParameterHelper parameterHelper,ILogger logger,CancellationToken cancellationToken) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 352
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.TryExecuteAsync(IFunctionInstance functionInstance,CancellationToken cancellationToken) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 108

2. It is regarding the grpcMaxMessageLength, there are multiple issues related to it. In Function Host Runtime, the limit should be set to int.MaxValue:

https://github.com/Azure/azure-functions-host/issues/2677

DefaultMaxMessageLengthBytes = int.MaxValue;
https://github.com/Azure/azure-functions-host/blob/dev/src/WebJobs.Script/Workers/Rpc/RpcWorkerConstants.cs#L38

3. From logs, we see that grpcMaxMessageLength has already been set to 2147483647. Howeve it does not seem to take place in the durable function.

Starting worker process with FileName:C:\Program Files\Java\microsoft-jdk-11.0.16.1/bin/java WorkingDirectory:C:\home\site\wwwroot Arguments: -XX:+TieredCompilation -XX:TieredStopAtLevel=1 -noverify -Djava.net.preferIPv4Stack=true -jar "C:\Program Files (x86)\SiteExtensions\Functions\4.19.0\workers\java\azure-functions-java-worker.jar" --host 127.0.0.1 --port 50628 --workerId ada93a63-8e10-49ac-ba64-19113424c536 --requestId d5533bd6-b6a2-442d-b394-22c67ae1f6b5 --grpcMaxMessageLength 2147483647

4. This issue can always be reproduced from local with latest function host version: 4.21.1.20667 when send a POST with payload larger than 4MB.

5. This issue cannot be reproduced when use Azure Durable Function for Python. Also, no issues with HTTP trigger in normal Java function app. Hence we believe it is only with Azure Durable Function for Java.

6. There is no such setting for "grpc max message size" in this repo found, it seems to inherit default 4MB limit in gRPC.