ytsaurus / ytsaurus/ytsaurus

YT QL ignores query settings (input_row_limit, output_row_limit, etc.)

Open
#1,405 0 comments 0 reactions 1 assignee Claimed by @Krisha11 View on GitHub
query tracker
Dominant language
C++
Stars
2.2k
Forks
219
PR merge metrics
No merged PRs in 30d

Description

## Description

When executing YT QL queries through Query Tracker, the settings passed in the request (like `input_row_limit`, `output_row_limit`, `fail_on_incomplete_result`) are completely ignored. This leads to queries being terminated prematurely with errors like:

```
Query terminated prematurely due to excessive input;
consider rewriting your query or changing input limit
```

Even when users explicitly set higher limits in the UI or API, these settings have no effect.

## Root Cause

The bug is in `/yt/yt/server/query_tracker/ql_engine.cpp`. The `TQLQueryHandler::Start()` method calls `SelectRows` without passing any options:

```cpp
// Current code (line 48):
AsyncQueryResult_ = QueryClient_->SelectRows(Query_);
```

The settings from the query are parsed but never converted to `TSelectRowsOptions` and passed to the `SelectRows` method.

## Expected Behavior

The settings specified in the Query Tracker request should be respected. The code should:
1. Parse settings like `input_row_limit`, `output_row_limit`, `fail_on_incomplete_result` from the query settings
2. Create a `TSelectRowsOptions` object with these values
3. Pass the options to `SelectRows`

## Proposed Fix

The `Start()` method in `TQLQueryHandler` should be modified to something like:

```cpp
void Start() override
{
YT_LOG_DEBUG("Starting QL query");
OnQueryStarted();

// Parse settings and create options
TSelectRowsOptions options;
auto settings = ConvertToAttributes(ActiveQuery_.Settings);

if (auto inputRowLimit = settings->Find("input_row_limit")) {
options.InputRowLimit = inputRowLimit;
}
if (auto outputRowLimit = settings->Find("output_row_limit")) {
options.OutputRowLimit = outputRowLimit;
}
if (auto failOnIncomplete = settings->Find("fail_on_incomplete_result")) {
options.FailOnIncompleteResult = failOnIncomplete;
}
// Add other settings as needed...

AsyncQueryResult_ = QueryClient_->SelectRows(Query_, options);
AsyncQueryResult_.Subscribe(BIND(&TQLQueryHandler::OnQueryFinish, MakeWeak(this)).Via(GetCurrentInvoker()));
}
```

## How to Reproduce

1. Create a dynamic table with a large amount of data
2. Run a YT QL query through Query Tracker UI or API with settings:
```json
{
"cluster": "your-cluster",
"input_row_limit": 10000000,
"fail_on_incomplete_result": false
}
```
3. Observe that the query still fails with "Query terminated prematurely due to excessive input" error, using the default limit instead of the specified one

## Impact

This bug makes it impossible to query large datasets using YT QL through Query Tracker, as users cannot override the default row limits. This severely limits the usability of YT QL for production queries.

## Environment

- Component: Query Tracker, YT QL Engine
- File: `/yt/yt/server/query_tracker/ql_engine.cpp`
- Affected versions: All current versions (checked in main branch)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.