ClickHouse / ClickHouse/clickhousectl

Establish supported Kafka schema-discovery authentication modes

Open
#841 0 comments 0 reactions 0 assignees View on GitHub
bug cloud documentation
Dominant language
Rust
Stars
74
Forks
5
Avg merge
2d 11h
Merged PRs (30d)
196

Description

## Validated scope and current-PR plan — 2026-09-11

This section records the QA review and the current decision to finalize existing PRs without opening new PRs. It supersedes the proposed routing/fix suggestions in the original report below; the original observations are retained. Central plan: #757.

### Work for the existing PRs

Implementation status: the checked sub-items below are implemented on the existing PR branches and validated locally on combined head `82b374816a917cbac3a68194659beaaa5c7c9087`. Required final-head Cloud CI and merging remain pending; this issue stays open. See #757 for the combined validation and CI status.

- [x] #775 — Expose canonical possible values on the remaining ClickPipe `--auth` flags. Preserve current accepted modes and aliases; auth inference and new credential requirements are outside this change.

### Remaining scope

Kafka no-auth behavior and Kinesis credential inference/requirements remain deferred; they are not part of choice visibility.

This is a partial assignment. Keep the issue open after these PRs land; use sub-item references rather than a whole-issue closing keyword.

### Validation and corrections

Confirmed object-storage --auth help lacks choices despite invalid input already producing exit 2. #775’s documented fixed-choice promise makes this a reasonable completion. Kafka no-auth rejection and Kinesis inference are different behavioral changes; establish allowed auth modes before changing defaults or requiring credentials. Do not put all of #841 into an enum-help PR.

---

## Original QA report

Found in the 2026-09-11 QA sweep of the combined PR stack at [`d2c37f81`](https://github.com/ClickHouse/clickhousectl/commit/d2c37f814fff53fe68d636e34103e2d0550bf84d), top PR #823, installed release build (package version still 0.4.2). Reproduced behaviour only; no implementation included.

Severity: medium; the exact gap #684 closed for the other closed-set flags.

| screen | `[possible values:]` in help | values in the error |
|---|---|---|
| `schema-discover kinesis`, `create kinesis`, `create postgres`, `create mysql` | yes | – |
| `create kafka`, `schema-discover kafka` | no | yes |
| `create object-storage`, `schema-discover object-storage` | no | **no** |
| `create pubsub`, `schema-discover pubsub` | no | **no** |
| `create bigquery` | no | **no** |

For object-storage, pubsub and bigquery the legal values appear nowhere in the CLI:

```sh
clickhousectl cloud clickpipe create object-storage "$SVC" … --auth NOPE
# error: invalid value 'NOPE' for '--auth ': unknown object-storage authentication method 'NOPE'
```

Kafka `--auth` help: "Inferred from the credential flags when omitted; with no credential flag at all, no authentication is sent." That is what it does, and the server rejects it every time with an internal JSON path:

```sh
clickhousectl cloud clickpipe schema-discover "$SVC" kafka --brokers b:9092 --topics t --format JSONEachRow
# Error: BAD_REQUEST: request body.source.kafka.authentication: 'undefined'
```

Pubsub validates the same situation locally (`--auth SERVICE_ACCOUNT requires --service-account-file`).

Kinesis is the one source where auth is *not* inferred: `--auth` defaults to `IAM_ROLE`, so passing `--access-key-id/--secret-key` alone fails at the server with `iamRole is required for IAM_ROLE authentication`.

Suggested: `PossibleValuesParser` (or `parse_serde_enum`) for every `--auth`; local "credential flag or `--auth` required" check for kafka mirroring pubsub; fix both kafka `--auth` doc comments; infer `IAM_USER` on kinesis when key flags are present or reject locally.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the existing ClickPipe --auth implementations and PR #775, then read the validation and combined-status discussion in #757. The remaining documented scope is exposing canonical allowed values while preserving accepted modes and aliases; authentication inference, credential requirements, Kafka no-auth rejection, and Kinesis changes are deferred.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
authentication, cli
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.