ClickHouse / ClickHouse/clickhouse-connect
Feature Request: Implement Progress Tracking for Long-Running Queries
- Dominant language
- Python
- Stars
- 521
- Forks
- 159
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 43
Description
### Description:
I would like to suggest the implementation of a progress tracking mechanism for long-running queries, such as `insert from S3`. This feature could be incredibly beneficial in monitoring the execution of these extensive operations.
### Motivation:
In many cases, queries in ClickHouse can take a substantial amount of time to execute, ranging from several minutes to hours or even days. During such long-running operations, users currently do not have a way to monitor the progress of these queries. Implementing a progress tracking feature, akin to what the ClickHouse CLI client offers, would be extremely beneficial. This would not only improve the user experience by providing real-time updates on query execution but also help in diagnosing and troubleshooting any issues that might arise during the execution of these lengthy queries.
Contributor guide
Research direction
No files, tests, or entry points are named. Start by surveying the Python driver's query execution path and how ClickHouse exposes progress for long-running operations; done would require a defined progress-tracking mechanism with real-time updates for operations such as insert from S3.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100