The serving surface: TLS, listeners, and interactive auth for a control plane that can face the internet

Flowstate's identity substrate is strong and its serving surface has not caught up. The result is a control plane that verifies tokens rigorously over a transport that cannot protect them, and the clearest evidence is that our own client already refuses to talk to it.

The finding that sets the priority

cmd/flow/credentials.go:39 reads FLOWSTATE_INSECURE_PLAINTEXT_TOKEN and, absent it, refuses to send a bearer credential to anything that is not loopback. That refusal is correct. What makes it a finding is the other half: grep finds no tls.Config, no ListenAndServeTLS, and no ACME anywhere under cmd/ or pkg/flowstate/v1/server/. The only TLS configuration in the tree is outbound — the egress dialer (netpolicy/netpolicy.go:235) and the Vault client (secrets/vault/vault.go:802).

So the server cannot offer TLS, and the client is right not to trust it. We have achieved a safe posture by making the product unusable off loopback rather than by making the transport secure. Every item below follows from closing that.

What is already built, so the scope stays honest

This is a serving-surface gap, not an auth gap. Already on main:

• Inbound authentication through connectrpc.com/authn v0.2.0, with the authenticator attaching a verified Principal (auth/connect.go), and a bearer token deliberately kept out of logs.
• An internal issuer: signing keys, algorithm choice, JWKS path, key rotation with retention, bounded assertion lifetime, and redaction on every containment shape (auth/issuer.go). Workload identity federation is real here, not aspirational.
• OIDC verification and exchange: discovery, JWKS fetch, federation, cloud and generic OAuth exchange (auth/discovery.go, jwks.go, federation.go, exchange_oauth.go, exchange_cloud.go).
• Policy over identity for secrets and role assumption.

What is missing is everything between a socket and that machinery.

1. No TLS, no mTLS, no ACME

The server listens with net.Listen("tcp", …) and serves plain HTTP. For a deployment behind a service mesh that is defensible; for anything else it means bearer tokens on the wire. A control plane that can start runs, read secrets policy and mint workload assertions is not a service to put on the internet in plaintext.

Three things belong here, in order of how much they buy: explicit cert and key configuration, which unblocks every managed-certificate story anyone already has; automatic ACME for the deployment that wants one flag and a hostname; and optional mTLS, which is worth more here than usual because a verified client certificate is another identity source, and #548 is already arguing for one vocabulary that every policy surface reads. A client certificate subject arriving as the same identity an egress rule gates on is the cohesive version of this, not a fourth parallel notion of who is calling.

2. One listener, and nothing else served on it

There is exactly one mux with exactly one route on it — the Connect handler (cmd/flow/serverdev.go:729). Two consequences.

The issuer computes a JWKSURL() that nothing serves. An assertion this platform mints therefore cannot be verified by a relying party, which is most of the point of minting it. JWKS is public by design and belongs on the public listener beside the RPC handler.

And there is no internal listener, so there is nowhere to put the things an operator expects on a separate port: health and readiness, pprof, and a metrics endpoint for a scraper — which matters more than it looks, because telemetry today is OTLP-push-only, and a Prometheus-shaped deployment has no way in. The shape people expect is well established: one public port carrying everything an external client or a flow CLI needs, and one internal port bound separately for scraping and debugging, never the same socket.

3. No interactive authorization, so humans and agents cannot get a token

golang.org/x/oauth2 is an indirect dependency. There is no authorization endpoint, no PKCE, no device authorization grant. A human running flow or an agent connecting over MCP must already possess a credential obtained by means flowstate knows nothing about, which pushes every deployment into hand-rolled token distribution — the failure mode where credentials end up in shell profiles and CI variables.

This is most acute for MCP. flow mcp is stdio and in-process today, so it inherits whatever the parent process has; the moment it is reachable over HTTP it needs the OAuth story MCP now expects, which includes protected-resource metadata pointing at an authorization server. The device authorization grant is the right shape for both a CLI on a headless box and an agent with no browser.

If flowstate acts as an authorization server at all, OAuth 2.1 is the floor and it is mostly a list of things not to implement: PKCE required on every authorization code flow, no implicit grant, no resource owner password grant, exact redirect URI matching. Being late here is an advantage — there is no legacy flow to keep working.

4. Bearer tokens are bearer tokens

Everything above still leaves a stolen token fully usable. Sender-constraining is the defense-in-depth answer and there are two standard ways to get it: DPoP (RFC 9449), which binds a token to a key the client proves possession of per request and works without mTLS; and mTLS-bound tokens (RFC 8705), which come nearly free where mutual TLS is already configured. These are additive and should be selectable rather than mandatory, since a deployment behind a mesh may reasonably decline both.

5. Injecting credentials at the proxy, not into the workflow

An upstream that needs a credential currently gets one through a task input resolved from a secret reference. The invariant is held — the value resolves inside the activity and never enters history — but the workflow still names the secret, which means authoring a workflow requires knowing which credential it will use.

The alternative is the same architectural move this repository already made twice: put it below the layer that could get it wrong. The egress dialer is already the enforcement point for network policy, and the byte cap already lives on the RoundTripper because a library option missed an error path. A credential-injecting egress proxy is that same seam: policy says this identity, reaching this host, gets this credential, the workflow says only https://partner-a.example.com/v1, and the material never enters the workflow's vocabulary at all. That is strictly stronger than redaction, because there is nothing to redact.

Sketches

Illustrative, not the landed shape. Proto-first, because this is exactly the configuration surface that turns into thirty flags if it is born as flags:

message ServerConfig {
  Listener public = 1;    // RPC, JWKS, OAuth endpoints, well-known metadata
  Listener internal = 2;  // health, metrics, pprof — a separate socket, always
  AuthConfig auth = 3;
}

message Listener {
  string address = 1 [(buf.validate.field).string.min_len = 1];
  TLSConfig tls = 2;      // absent means plaintext, which is refused unless
                          // the address is loopback
}

message TLSConfig {
  oneof source {
    FileCertificate file = 1;
    ACMEConfig acme = 2;       // Let's Encrypt or any ACME directory
  }
  // Present means mutual TLS is required. A verified client certificate
  // becomes an identity the same policy vocabulary reads (#548), not a
  // fourth parallel notion of who is calling.
  ClientAuthConfig client_auth = 3;
  string min_version = 4;      // "1.3" by default, never below "1.2"
}

message AuthConfig {
  // Sender constraining. Both are additive and neither is required: a
  // deployment behind a mesh may decline both, and must say so rather
  // than get it by omission.
  bool require_dpop = 1;              // RFC 9449
  bool require_mtls_binding = 2;      // RFC 8705

  // Interactive grants this deployment offers. OAuth 2.1: PKCE is not a
  // toggle, it is required wherever authorization_code appears.
  repeated Grant grants = 3;          // authorization_code, device_code
}

what an operator writes, which should be short in the common case:

server:
  public:
    address: ":443"
    tls:
      acme:
        hosts: [flowstate.example.com]
        cache: /var/lib/flowstate/acme
  internal:
    address: "127.0.0.1:9090"
  auth:
    require_dpop: true
    grants: [device_code]

and what an agent or a headless CLI does to get a credential at all:

$ flow login --server https://flowstate.example.com
open https://flowstate.example.com/device and enter code  WDJB-MJHT
waiting…
logged in as alice@example.com  (expires in 8h, DPoP-bound)

Where the pieces sit:

flowchart LR
    H[human / agent] -->|device code + PKCE| AS[authorization endpoints<br/>public listener]
    H -->|DPoP-bound bearer| RPC[Connect RPC<br/>public listener]
    RP[relying party] -->|verify assertion| JWKS[/.well-known/jwks.json<br/>public listener/]
    SCR[scraper] --> INT[metrics + health<br/>internal listener]
    RPC --> W[worker]
    W -->|no credential in the spec| PX[egress proxy<br/>injects per policy]
    PX --> UP[upstream]

Constraints

• Fail closed, including on the new surfaces. A configured hostname whose certificate cannot be obtained is a startup failure. Plaintext on a non-loopback address is refused rather than warned about — the client already takes this position, and the server should agree with it.
• Proto-first. This is configuration describing a deployment, it travels, and it will grow; it is a schema, not a flag list. Flags remain for the handful of things a person types interactively.
• One vocabulary for identity. A client certificate subject, an OIDC subject and a workload assertion are three sources of one thing, and #548's unification is a prerequisite rather than a parallel effort.
• Nothing new in history or logs. DPoP proofs, client certificates and injected credentials all sit on paths that must not reach Temporal history, with the containment shapes tested as invariant 7 requires.
• Bound every new parser. ACME responses, DPoP proofs, JWKS documents and client certificate chains are all attacker-influenced input reached before authentication.

Questions

1. Slice order. Recommended: TLS with explicit cert/key plus the internal listener and JWKS first, since that is the smallest change that lets the client stop refusing and gives operators a scrape target. ACME second, mTLS third, interactive grants fourth. Reasonable to swap ACME earlier if the target is a single-hostname deployment.
2. Does flowstate become an authorization server, or only a resource server? Recommended resource-server-first: publish protected-resource metadata, verify tokens from an external IdP, and add device-code issuance only if deployments without an IdP turn out to be common. Being an AS is a large, permanent obligation.
3. DPoP or mTLS binding first? Recommended DPoP, because it works for browserless agents without requiring the mTLS deployment story to land first.
4. Is the credential-injecting proxy this issue's scope or its own? Recommended its own, hanging off #548's enforcement seam, because it is a policy mechanism that happens to involve transport rather than a transport feature.

Related: #548 (one policy vocabulary, and the enforcement-below-the-workload principle this reuses), #547 (the internal listener is also where a scrape target would live).

Start at cmd/flow/serverdev.go:729 to trace the current listener and Connect mux, then read cmd/flow/credentials.go:39 to understand the client's plaintext refusal. Begin with the recommended first slice: explicit TLS configuration, a separate internal listener, and a served JWKS endpoint. It is done when non-loopback plaintext fails closed, the client can connect securely, and internal health, metrics, and debugging surfaces are isolated.