YT QL ignores query settings (input_row_limit, output_row_limit, etc.)
- Dominant language
- C++
- Stars
- 2.2k
- Forks
- 219
- PR merge metrics
- No merged PRs in 30d
Description
## Description
When executing YT QL queries through Query Tracker, the settings passed in the request (like `input_row_limit`, `output_row_limit`, `fail_on_incomplete_result`) are completely ignored. This leads to queries being terminated prematurely with errors like:
```
Query terminated prematurely due to excessive input;
consider rewriting your query or changing input limit
```
Even when users explicitly set higher limits in the UI or API, these settings have no effect.
## Root Cause
The bug is in `/yt/yt/server/query_tracker/ql_engine.cpp`. The `TQLQueryHandler::Start()` method calls `SelectRows` without passing any options:
```cpp
// Current code (line 48):
AsyncQueryResult_ = QueryClient_->SelectRows(Query_);
```
The settings from the query are parsed but never converted to `TSelectRowsOptions` and passed to the `SelectRows` method.
## Expected Behavior
The settings specified in the Query Tracker request should be respected. The code should:
1. Parse settings like `input_row_limit`, `output_row_limit`, `fail_on_incomplete_result` from the query settings
2. Create a `TSelectRowsOptions` object with these values
3. Pass the options to `SelectRows`
## Proposed Fix
The `Start()` method in `TQLQueryHandler` should be modified to something like:
```cpp
void Start() override
{
YT_LOG_DEBUG("Starting QL query");
OnQueryStarted();
// Parse settings and create options
TSelectRowsOptions options;
auto settings = ConvertToAttributes(ActiveQuery_.Settings);
if (auto inputRowLimit = settings->Find("input_row_limit")) {
options.InputRowLimit = inputRowLimit;
}
if (auto outputRowLimit = settings->Find("output_row_limit")) {
options.OutputRowLimit = outputRowLimit;
}
if (auto failOnIncomplete = settings->Find("fail_on_incomplete_result")) {
options.FailOnIncompleteResult = failOnIncomplete;
}
// Add other settings as needed...
AsyncQueryResult_ = QueryClient_->SelectRows(Query_, options);
AsyncQueryResult_.Subscribe(BIND(&TQLQueryHandler::OnQueryFinish, MakeWeak(this)).Via(GetCurrentInvoker()));
}
```
## How to Reproduce
1. Create a dynamic table with a large amount of data
2. Run a YT QL query through Query Tracker UI or API with settings:
```json
{
"cluster": "your-cluster",
"input_row_limit": 10000000,
"fail_on_incomplete_result": false
}
```
3. Observe that the query still fails with "Query terminated prematurely due to excessive input" error, using the default limit instead of the specified one
## Impact
This bug makes it impossible to query large datasets using YT QL through Query Tracker, as users cannot override the default row limits. This severely limits the usability of YT QL for production queries.
## Environment
- Component: Query Tracker, YT QL Engine
- File: `/yt/yt/server/query_tracker/ql_engine.cpp`
- Affected versions: All current versions (checked in main branch)
Contributor guide
Assessment
This issue has not been assessed yet.