Tuning required for dropping queries on server-side after async-broker change

With async-broker, it is observed that the broker keeps piling queries on the server side asynchronously even if the server is struggling. Since the server doesn't have a good way to preempt queries in flight, this leads to a request buildup on the server side, and makes it harder for server to recover for high qps use cases.

Prior to async-broker, the broker to server connections would be blocked for queries in flight, and so the query buildup would happen on broker side until a point at which broker starts rejecting queries from the client. This is not desirable either.

One way to mitigate this would be if the server becomes smarter in dropping queued (or inflight ) queries for which SLA is already missed. For example, if the query is already been in the queue for longer than its SLA, then there is no point executing that query.