ClickHouse / ClickHouse/clickhouse-connect

Feature Request: Implement Progress Tracking for Long-Running Queries

Open
#285 5 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
521
Forks
159
Avg merge
1d 8h
Merged PRs (30d)
43

Description

### Description:

I would like to suggest the implementation of a progress tracking mechanism for long-running queries, such as `insert from S3`. This feature could be incredibly beneficial in monitoring the execution of these extensive operations.

### Motivation:
In many cases, queries in ClickHouse can take a substantial amount of time to execute, ranging from several minutes to hours or even days. During such long-running operations, users currently do not have a way to monitor the progress of these queries. Implementing a progress tracking feature, akin to what the ClickHouse CLI client offers, would be extremely beneficial. This would not only improve the user experience by providing real-time updates on query execution but also help in diagnosing and troubleshooting any issues that might arise during the execution of these lengthy queries.

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by surveying the Python driver's query execution path and how ClickHouse exposes progress for long-running operations; done would require a defined progress-tracking mechanism with real-time updates for operations such as insert from S3.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.