apache / apache/druid

Scan query gracefully closing connections which results in partial data read from datasource

Open
#11,422 1 comment 0 reactions 0 assignees View on GitHub
Area - Querying
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

### Affected Version

Druid 0.19.1

### Description

The scenario arises when we fire a Scan query to read all rows for a given day (typically around 1mill records, takes around 10-15 min to complete) from the datasource.

We observe that if there are N records for that given day, sometimes it will read

Contributor guide

Open the contributing guide

Research direction

Review the closed pull request #13492 and the reported broker, historical, and router timeout configurations first, since the issue links no source files or tests. Reproduce the parallel Scan-query stress test described in the report and verify that a complete result is returned without silent connection closure or partial data.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.