ClickHouse / ClickHouse/clickhousectl

Explain the GCP scope of ClickPipe workload identity context

Open
#844 0 comments 0 reactions 0 assignees View on GitHub
bug cloud
Dominant language
Rust
Stars
74
Forks
5
Avg merge
2d 11h
Merged PRs (30d)
196

Description

## Validated scope and current-PR plan — 2026-09-11

This section records the QA review and the current decision to finalize existing PRs without opening new PRs. It supersedes the proposed routing/fix suggestions in the original report below; the original observations are retained. Central plan: #757.

### Work for the existing PRs

Implementation status: the checked sub-items below are implemented on the existing PR branches and validated locally on combined head `82b374816a917cbac3a68194659beaaa5c7c9087`. Required final-head Cloud CI and merging remain pending; this issue stays open. See #757 for the combined validation and CI status.

- [x] #811 — Resolve item 6 (hidden empty fields) through the shared renderer; no separate ClickPipe-specific rendering change.

- [x] #814 — Resolve item 1: remove serde Option/untagged traversal artifacts from generic unknown-field paths and identify stdin input clearly. Retain strict unknown-field rejection and the specific BigQuery diagnostic.

### Remaining scope

GCP-context guidance, not-found rewrites (#831), delete-name output (#843) and JSON casing policy (#863) remain deferred.

This is a partial assignment. Keep the issue open after these PRs land; use sub-item references rather than a whole-issue closing keyword.

### Validation and corrections

Reproduced source.?.postgres.?.settings.?.initialLoadParallelism and “config -”. #814 is the direct missing follow-up: it special-cases BigQuery without fixing generic serde path rendering. Item 3 shares #831; item 4 shares #843. Item 2 is new GCP-context guidance. Item 5 belongs to #863’s contract discussion: snake_case settings are API wire names, not an accidental CLI rename. Item 6 duplicates #850.

---

## Original QA report

Found in the 2026-09-11 QA sweep of the combined PR stack at [`d2c37f81`](https://github.com/ClickHouse/clickhousectl/commit/d2c37f814fff53fe68d636e34103e2d0550bf84d), top PR #823, installed release build (package version still 0.4.2). Reproduced behaviour only; no implementation included.

Severity: low-medium; batch of output defects in `clickpipes.rs`.

1. PATCH validation leaks `Option`/untagged-enum traversal into the field path:
`unknown field \`source.?.postgres.?.settings.?.initialLoadParallelism\``. Sibling messages are clean (`\`source.mysql\` is missing required field \`port\``). Also "in config -" when the body came from stdin. Strip `?` segments; say "config read from stdin".
2. `clickpipe context get` on any AWS service → `Error: BAD_REQUEST: GCP workload identity authentication is not available`, exit 1. The `about` ("Get service capabilities and workload identity") and the `create` CONTEXT block point users at it with no hint it is GCP-only. Scope the about or rewrite that 400.
3. `clickpipe delete "$SVC" "$GONE"` → `Error: NOT_FOUND: 536e7444-…` — a bare UUID. `get` on a well-formed unknown UUID → `Invalid clickpipe id. Expect a valid UUID, given: ` (doubled space, wrong claim). Rewrite to "ClickPipe not found".
4. `delete` prints the ID (`ClickPipe 536e7444-… deleted`) where `stop`/`start`/`resync` print the name; the delete response carries the name.
5. `clickpipe get/list/update/scale/cdc-scaling --json` emit camelCase (mirroring the API); `clickpipe settings get/update --json` emit snake_case (`object_storage_max_insert_bytes`). `get` also returns the same `settings` object in camelCase, so the same data has two spellings depending on the command.
6. Human `get` drops `fieldMappings: []`, `destination.table: ""`, `destination.columns: []` that JSON shows, while showing `settings: {}` (#806) — see #850.

Contributor guide

No contributing guide indexed for this repository

Research direction

The report points to clickpipes.rs and the clickpipe context get/about/create, delete, and get command paths. Start by reading the current validation and scope in #757, then check the remaining GCP-context item against the related work in #831, #843, and #863. Done means an explicitly assigned remaining item is validated without duplicating work already implemented in the existing PRs.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.