ClickHouse / ClickHouse/clickhousectl
Verify source-kind rules before enforcing ClickPipe field mappings
- Dominant language
- Rust
- Stars
- 74
- Forks
- 5
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 196
Description
## Validated scope and current-PR plan — 2026-09-11
This section records the QA review and the current decision to finalize existing PRs without opening new PRs. It supersedes the proposed routing/fix suggestions in the original report below; the original observations are retained. Central plan: #757.
### Work for the existing PRs
Implementation status: the checked sub-items below are implemented on the existing PR branches and validated locally on combined head `82b374816a917cbac3a68194659beaaa5c7c9087`. Required final-head Cloud CI and merging remain pending; this issue stays open. See #757 for the combined validation and CI status.
- [x] #793 — Bring the already-verified object-storage fieldMappings/destination.columns requirement into update input help. Keep it scoped to object-storage; do not introduce unverified universal PATCH validation.
### Remaining scope
Client-side mapping enforcement remains deferred until the rule is verified for the relevant pipe kinds; #793 explicitly avoided assuming a universal rule. CDC pause guidance is also outside this narrow documentation completion.
This is a partial assignment. Keep the issue open after these PRs land; use sub-item references rather than a whole-issue closing keyword.
### Validation and corrections
Reproduced an HTTP request for {fieldMappings: []} without columns. However #793’s body explicitly says it avoided global validation because source kind is commonly omitted, and only object-storage behavior was verified. #813 validates source ports; it does not own all PATCH invariants. New enforcement must either establish a universal rule or obtain/validate the existing pipe kind. Preserve omission semantics and cover explicit []. CDC pause guidance is also valid new help work; do not silently pause/resume the pipe.
---
## Original QA report
Found in the 2026-09-11 QA sweep of the combined PR stack at [`d2c37f81`](https://github.com/ClickHouse/clickhousectl/commit/d2c37f814fff53fe68d636e34103e2d0550bf84d), top PR #823, installed release build (package version still 0.4.2). Reproduced behaviour only; no implementation included.
Severity: low-medium.
```sh
printf '{"source":{"postgres":{"settings":{"syncIntervalSeconds":30}}}}' | clickhousectl cloud clickpipe update "$SVC" "$CDC" --config-file -
# Error: BAD_REQUEST: Postgres ClickPipe must be paused to edit (succeeds after `clickpipe stop`)
printf '{"fieldMappings":[{"sourceField":"price","destinationField":"price"}]}' | … update … --config-file -
# Error: BAD_REQUEST: Columns are required if field mappings are provided
printf '{"fieldMappings":[]}' | … update … --config-file -
# Error: BAD_REQUEST: Columns are required if field mappings are provided (explicit [] rejected too)
```
The second matches the README text #763 added exactly — README is correct. But it lives only in the README and is not enforced in `validate_clickpipe_patch_required_fields`, which already runs structural PATCH checks for mysql/mongodb/pubsub/bigquery in the same function. Neither the `--config-file` doc nor the 4-line CONTEXT block mentions pausing CDC pipes.
Suggested: add the `fieldMappings ⇒ destination.columns` check (including explicit `[]`) next to the sibling checks; one CONTEXT line: "CDC pipes must be stopped (Paused) before `update`."
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with validate_clickpipe_patch_required_fields and the existing README and --config-file documentation referenced in the report. Read #757, #793, and #813 first to understand the verified source-kind scope and existing work. Done means the remaining documentation or validation scope is completed without assuming a universal rule, while preserving omission semantics and handling explicit empty mappings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli, documentation
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100