stackabletech / stackabletech/trino-lb
Reduce impact of ExplainCostsRouter and many concurrent queries
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 37
- Forks
- 8
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 1
Description
Currently 100,000 queries in trino-lb running through ExplainCostsRouter cause 100,000 queries on the underlying Trino cluster used for the query estimations, which can lead to DOS.
To prevent this we need to let users specify the cluster group used for the explain queries instead of a single cluster. This should also take care of queuing explain queries.
An alternative solution could be to enter trino-lb itself as the explain target cluster, but users would need to be careful to not creating routing loops 🤷♂️ To prevent this a dedicated trino-lb only for explain queries could be set up. Seems unnecessary complicated though.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing ExplainCostsRouter and the existing trino-lb explain-query routing and cluster configuration entry points. Determine how users can select a cluster group and how explain queries are queued without creating routing loops. Done means explain traffic no longer sends every request to one underlying cluster and concurrent queries are protected from causing a denial of service.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100