Integration test with nUnit seems to fail after .net core has been upgraded from 3.1 to 5

I have recently upgraded my .net core version from 3.1.8 to 5, my web API seems to work fine, however integration test for the scenario where API returns notFound status code along with custom message are failing with below error, and the failure occur randomly.

Message:
System.IO.IOException : No message provided

Stack Trace:
ResponseBodyReaderStream.CheckAborted()
ResponseBodyReaderStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
ResponseBodyReaderStream.Read(Byte[] buffer, Int32 offset, Int32 count)
DelegatingStream.Read(Byte[] buffer, Int32 offset, Int32 count)
StreamReader.ReadBuffer(Span1 userBuffer, Boolean& readToUserBuffer) StreamReader.ReadSpan(Span1 buffer)
StreamReader.Read(Char[] buffer, Int32 index, Int32 count)
JsonTextReader.ReadData(Boolean append, Int32 charsRequired)
JsonTextReader.ParseValue()
JsonReader.ReadAndMoveToContent()
JsonReader.ReadForType(JsonContract contract, Boolean hasConverter)
JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
JsonSerializer.Deserialize(JsonReader reader, Type objectType)
BaseJsonMediaTypeFormatter.ReadFromStream(Type type, Stream readStream, Encoding effectiveEncoding, IFormatterLogger formatterLogger)
JsonMediaTypeFormatter.ReadFromStream(Type type, Stream readStream, Encoding effectiveEncoding, IFormatterLogger formatterLogger)
BaseJsonMediaTypeFormatter.ReadFromStream(Type type, Stream readStream, HttpContent content, IFormatterLogger formatterLogger)
BaseJsonMediaTypeFormatter.ReadFromStreamAsync(Type type, Stream readStream, HttpContent content, IFormatterLogger formatterLogger)
--- End of stack trace from previous location ---
HttpContentExtensions.ReadAsAsyncCore[T](HttpContent content, Type type, IFormatterLogger formatterLogger, MediaTypeFormatter formatter, CancellationToken cancellationToken)
CustomerControllerTests.Delete_DeleteCustomer_CustomerIdNotFound_ReturnsNotFound() line 440
GenericAdapter1.BlockUntilCompleted() line 95 NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter awaiter) line 61 AsyncToSyncAdapter.Await(Func1 invoke) line 57
TestMethodCommand.RunTestMethod(TestExecutionContext context) line 81
TestMethodCommand.Execute(TestExecutionContext context) line 64
<>c__DisplayClass4_0.b__0() line 77
<>c__DisplayClass1_01.b__0(Object _) line 40 ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state) --- End of stack trace from previous location --- ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state) ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) ContextUtils.DoIsolated(ContextCallback callback, Object state) line 55 ContextUtils.DoIsolated[T](Func1 func) line 40
SimpleWorkItem.PerformWork() line 77
--HttpStatusException
CustomersService.Delete(Guid id, Byte[] rowVersion) line 271
CustomerController.Delete(DeleteRequest request) line 266
lambda_method903(Closure , Object )
AwaitableObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
ControllerActionInvoker.g__Logged|12_1(ControllerActionInvoker invoker)
ControllerActionInvoker.g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
ControllerActionInvoker.InvokeInnerFilterAsync()
--- End of stack trace from previous location ---
ResourceInvoker.g__Awaited|24_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
ResourceInvoker.InvokeFilterPipelineAsync()
--- End of stack trace from previous location ---
ResourceInvoker.g__Logged|17_1(ResourceInvoker invoker)
EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
AuthorizationMiddleware.Invoke(HttpContext context)
AuthenticationMiddleware.Invoke(HttpContext context)
RequestLocalizationMiddleware.Invoke(HttpContext context)
ExceptionHandlerMiddleware.g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, Task task)
ExceptionHandlerMiddleware.HandleException(HttpContext context, ExceptionDispatchInfo edi)
ExceptionHandlerMiddleware.g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, Task task)
CorrelationIdMiddleware.Invoke(HttpContext context, ICorrelationContextFactory correlationContextFactory)
d.MoveNext()__

I am created my testClient using below code:

TestClient = new TestWebApplicationFactory()
.CreateClient(new WebApplicationFactoryClientOptions
{
AllowAutoRedirect = false,
});