ClickHouse / ClickHouse/clickhousectl

Distinguish Query API wake delays from gateway execution timeouts

Open
#826 0 comments 0 reactions 0 assignees View on GitHub
bug cloud
Dominant language
Rust
Stars
74
Forks
5
Avg merge
2d 11h
Merged PRs (30d)
196

Description

## Validated scope and current-PR plan — 2026-09-11

This section records the QA review and the current decision to finalize existing PRs without opening new PRs. It supersedes the proposed routing/fix suggestions in the original report below; the original observations are retained. Central plan: #757.

### Disposition

Deferred from this existing-PR finalization pass. Keep the issue open; no new PR is requested now. This is not a resolution, a severity downgrade, or a waiver of release acceptance.

### Validation and corrections

Source supports the wake/timeout distinction, but the live idle-service failure was not rerun. Keep the diagnosis fix. Never automatically replay arbitrary SQL after an ambiguous timeout; it may already have executed. Distinguish pre-execution idle/wake responses from a query that has been submitted. “Every idle service” is stronger than two reproductions establish.

---

## Original QA report

Found in the 2026-09-11 QA sweep of the combined PR stack at [`d2c37f81`](https://github.com/ClickHouse/clickhousectl/commit/d2c37f814fff53fe68d636e34103e2d0550bf84d), top PR #823, installed release build (package version still 0.4.2). Reproduced behaviour only; no implementation included.

Severity: high; hits every idle-scaled service on first use.

```sh
clickhousectl cloud service get "$SVC" --json | jq -r .state # idle
time clickhousectl cloud service query --id "$SVC" -q 'SELECT 1'
# Error: the query timed out at the Query API gateway, which stops waiting after about 30 seconds.
# The statement may still be running on the service; check `SELECT query_id, elapsed FROM system.processes` …
# Hint: run long statements over the native protocol instead. …
# exit 1, ~60 s
clickhousectl cloud service query --id "$SVC" -q 'SELECT 1' # 2 s later: prints 1, exit 0
```

Reproduced twice. `query --help` says "An idle service wakes automatically" and `start --help` says "an idle service wakes on its own when queried". The wake *is* triggered (the immediate retry succeeds), but the CLI gives up while it is in flight and misattributes the failure to the 30-second gateway timeout, pointing the user at `system.processes` and a native-client fallback that are irrelevant for `SELECT 1`.

Expected: either poll/retry the wake transparently until the service reports `running` (that is the documented contract), or fail fast with a `service_idle`-style error saying the service is waking and to retry shortly. Anything but a wrong diagnosis.

Acceptance: wiremock test for the idle → wake → success path; the gateway-timeout message is never emitted when the pre-query state was `idle`.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the cloud service query command and its idle-service handling, then review the existing Wiremock test coverage. Reproduce the idle → wake → success sequence and add coverage for it; done means the gateway-timeout message is not emitted when the pre-query state was idle.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.