Performance on bursts of requests on IIS less than ideal and results in rejected requests
- Dominant language
- C#
- Stars
- 38.4k
- Forks
- 10.9k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 281
Description
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Describe the bug
We've have several complaints about the performance of requests with a little latency (e.g `Task.Delay(5000)`) on IIS result in more rejected requests than when running on System.Web on .NET Framework. After some digging the difference between the 2 implementations is how completions are handled with respect to resuming the IIS pipeline. IIS has 2 ways to resume the state machine that runs the module pipeline:
- [IndicateCompletion](https://docs.microsoft.com/en-us/iis/web-development-reference/native-code-api-reference/ihttpcontext-indicatecompletion-method)
- [PostCompletion](https://docs.microsoft.com/en-us/iis/web-development-reference/native-code-api-reference/ihttpcontext-postcompletion-method)
`IndicateCompletion` resumes the IIS state machine directly on the current thread and `PostCompletion` runs the state machine on an IIS thread re-entering the pipeline when that work item is picked up off the IIS thread pool. The System.Web implementation tries its best to stay on the same managed thread as an optimization and avoids posting the completion back into IIS's IOCP.
What we end up with is new requests being queued into the same place the continuations for existing requests which causes enough of a delay that the HTTP.sys queue can fill up and reject requests. Here's an illustration of what the flow looks like for both System.Web and ANCM.

### Expected Behavior
We should use `IndicateCompletion` on a new managed thread pool thread and avoid posting back to IIS's IOCP via `PostCompletion`. This should reduce the latency and avoid IIS getting delayed when trying to grabbing requests from HTTP.sys.
### Steps To Reproduce
_No response_
### Exceptions (if any)
_No response_
### .NET Version
_No response_
### Anything else?
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.