LDAPS/TLS auth-proxy deployment mode: standalone ch-oauth-ldap on customer premises for any ClickHouse's native LDAP auth

Summary

Add a second deployment mode for ch-oauth-ldap: a standalone OAuth→LDAP auth proxy that runs on customer premises (VM, bare metal, or a plain container host) and speaks LDAPS/TLS, so any ClickHouse — not only one inside an Altinity Cloud / Kubernetes environment — can use its native LDAP authentication and role mapping against OAuth/OIDC identities over an ordinary network.

This is deliberately not the in-cluster mode shipped under #19 (environment-level ClusterIP Service, plain LDAP on 389, default-on NetworkPolicy, accepted plaintext-in-cluster exception in ADR #16 Amendments). That mode assumes a trusted cluster network between ClickHouse and the helper. The auth-proxy mode assumes the opposite: the ClickHouse → helper hop crosses a network the operator does not control end to end, so transport confidentiality of the bearer token is mandatory — exactly the case ADR #16's transport requirement was written for, and the "exit (a)" named in its amendment.

Motivation

• Customers running ClickHouse outside Altinity Cloud (self-managed, other clouds, on-prem) want OAuth/OIDC users with dynamic roles without changing ClickHouse: ClickHouse's LDAP external user directory is already there and already supports TLS (<enable_tls>, <tls_require_cert>, CA/cert/key settings).
• Today the JWT travels as the LDAP simple-bind password. Off-cluster that must be LDAPS (or StartTLS) or it is a credential leak in transit.
• One helper instance (or a small HA pair behind the customer's own load balancer) can serve many ClickHouse clusters.

Deployment shape

IdP (Auth0 / OIDC)  <--JWKS-->  ch-oauth-ldap (auth proxy, LDAPS :636)  <--LDAPS-->  ClickHouse (any location)

• Standalone binary / container, configured by the existing YAML plus a new tls: section; runs under systemd or a container runtime; no Kubernetes objects, no NetworkPolicy — the security boundary is TLS, optionally mutual TLS.
• ClickHouse points <ldap_servers><oauth_helper><host> at the proxy's DNS name with <port>636</port>, <enable_tls>yes</enable_tls>, <tls_require_cert>demand</tls_require_cert> and <tls_ca_cert_file> (or a system CA), optionally <tls_cert_file>/<tls_key_file> for client authentication — the rest of the working config (bind DN template, role_mapping, verification_cooldown 0, search_limit) is unchanged from integration/clickhouse/clickhouse/common/config.d/ldap.xml.

Scope

• LDAPS listener (implicit TLS on connect, default port 636) in internal/ldap / cmd/ch-oauth-ldap: server certificate + key from files, PEM; minimum TLS 1.2 (default 1.3 if ClickHouse's client supports it — verify), sane cipher policy, no plaintext fallback on the LDAPS port; certificate/key reload on SIGHUP or file change without dropping the process; clear startup failure on a bad chain.
• Optional mutual TLS: verify ClickHouse's client certificate against a configured CA (tls.client_ca_file, tls.require_client_cert), so only enrolled ClickHouse nodes can even attempt a Bind.
• StartTLS on 389: evaluate whether ClickHouse's enable_tls=starttls is worth supporting or whether LDAPS-only is the right first cut (recommendation: LDAPS only; document the choice).
• Plaintext mode stays available but explicit: the current in-cluster deployment keeps working unchanged; a proxy started without a tls: section must refuse to bind a non-loopback address unless an explicit --allow-plaintext / listen_plaintext: true opt-in is set (fail closed by default for the off-cluster mode).
• Packaging for premises: sample systemd unit, sample config, container run example, cert layout, health/readiness on a separate loopback endpoint (no HTTP on the LDAP port), log/metrics guidance. Keep the redaction inventory green — TLS handshake errors must not log peer-supplied bytes.
• HA: document running N instances behind the customer's TCP load balancer (the connection-local session model already needs no shared state — the phase-5 Docker HA fixture is the proof); ClickHouse itself only takes one <host> per server definition.
• Docs: a new section in docs/ch-oauth-ldap-operator-guide.md for the auth-proxy mode (trust boundary, cert rotation runbook, ClickHouse TLS config copied from a working fixture), and an ADR #16 note that this mode satisfies the transport requirement without an exception.

Non-goals

• Changing the in-cluster mode's defaults or its NetworkPolicy story (#19 / ADR #16 Amendments stay as recorded).
• Terminating TLS in a sidecar/ingress in front of the helper (that just moves the plaintext hop); the helper terminates TLS itself.
• Any change to the OAuth/JWT verification path or the role pipeline.
• Kubernetes packaging of the TLS mode (can follow later as a chart option; not required for premises use).

Tests

• internal/ldap: TLS listener tests (handshake with a test CA; plaintext client on the LDAPS port is rejected before any LDAP bytes are parsed; mTLS accept/reject; cert reload; TLS 1.0/1.1 refused).
• Redaction: extend the internal/securitytest inventory to the new TLS code paths; marker tests that a hostile ClientHello/SNI never reaches logs.
• Integration: a fixture variant (integration/clickhouse, opt-in profile) running ClickHouse with enable_tls=yes, tls_require_cert=demand, a test CA, and mTLS against the proxy — the existing scenarios A–I and G' must pass unchanged over LDAPS; the leak scan must stay clean.
• HA: run-ha.sh variant over LDAPS (HAProxy in TCP passthrough mode).

Open questions

• Does the customer typically have a PKI, or should the proxy be able to bootstrap a private CA + issue a ClickHouse client cert for the mTLS case (probably out of scope; document how to do it with openssl/step)?
• Do we need StartTLS at all, or is LDAPS sufficient for every ClickHouse version we track (24.8+)?
• Certificate rotation expectations (ACME/cert-manager on premises vs manual) — informs whether file-watch reload is enough.

Definition of done

A ClickHouse instance outside any Altinity/Kubernetes environment, configured only through its native <ldap_servers>/<user_directories><ldap> XML with enable_tls=yes (and optionally a client certificate), authenticates an OAuth user and receives the mapped roles through a ch-oauth-ldap auth proxy running on a plain host; a packet capture on the ClickHouse → proxy hop shows no plaintext bearer; a plaintext client cannot connect to the LDAPS port; the proxy refuses to start in plaintext on a non-loopback address without an explicit opt-in; all existing gates (Go, internal/securitytest, chart, integration, HA) stay green.

Related: #19 (in-cluster mode and its accepted plaintext exception), #16 (ADR; this mode is the "exit (a)" its amendment names), #30 (ClickHouse version compatibility).

Start by reading internal/ldap and cmd/ch-oauth-ldap, then inspect integration/clickhouse fixtures and the existing common/config.d/ldap.xml. Review ADR #16 and the phase-5 Docker HA fixture before resolving the StartTLS and certificate-rotation questions. Done means the named TLS, mTLS, redaction, integration, HA, packaging, and documentation checks pass, including the plaintext rejection and external ClickHouse authentication criteria.