In the load pipeline, separate the network request from the post-download CPU work, and throttle them differently

I am not sure how much of an impact it has, but our load thread pool is usually saturated by long network requests. Tiles that can be retrieved from the SQL cache however are extremely fast. This creates a conflict where cache retrievals are inevitably waiting on a load thread to finish resolving a network request. I think it ends up being bad scheduling to have very long-running tasks and very fast tasks in the same threads. If we can dedicate a thread pool to only process cache hits, cache loads may become even faster. The network load pool and cache load pool may need to be dynamically sized to not introduce any new bottlenecks.

The issue names no files, tests, or entry points. Start by tracing the load pipeline and its SQL-cache and network-request paths, then determine how separate pools and dynamic sizing would be coordinated; done means cache retrievals no longer wait behind long network tasks without introducing a new bottleneck.

Written by the indexing model from the issue text.