Vault Agent: auto-auth self-healing not wired for proxy/cache requests (missing channel assignment in `command/agent.go`)
- Dominant language
- Go
- Stars
- 36.3k
- Forks
- 4.8k
- PR merge metrics
- PR metrics pending
Description
Hi team,
I'd like to report what appears to be a missing channel assignment in `command/agent.go` that prevents the auto-auth self-healing feature from working when Vault Agent is used with the caching/proxy listener. I wanted to check with you whether this is intentional (given Agent's API proxy is deprecated in favor of Vault Proxy) or if it was an oversight.
## Summary
The auto-auth self-healing feature (introduced in v1.17.0 via [#26307](https://github.com/hashicorp/vault/pull/26307)) correctly wires the `invalidTokenErrCh` and `authInProgress` channels in `command/proxy.go`, but the equivalent assignment is missing in `command/agent.go`. This means the self-healing works for `vault proxy` but not for `vault agent` when using the cache/listener.
## Code References (v1.21.4)
In **`command/proxy.go`**, the local variables are reassigned to the auth handler's channels after `ah` is created:
https://github.com/hashicorp/vault/blob/v1.21.4/command/proxy.go#L578-L579
```go
authInProgress = ah.AuthInProgress
invalidTokenErrCh = ah.InvalidToken
```
In **`command/agent.go`**, the same variables are created at:
https://github.com/hashicorp/vault/blob/v1.21.4/command/agent.go#L552-L553
```go
authInProgress := &atomic.Bool{}
invalidTokenErrCh := make(chan error)
```
But they are **never reassigned** to `ah.AuthInProgress` / `ah.InvalidToken`. They are passed directly to `ProxyHandler` as orphaned local variables:
https://github.com/hashicorp/vault/blob/v1.21.4/command/agent.go#L685-L687
For reference, the auth handler creates its own buffered channel and atomic bool that are meant to be used:
https://github.com/hashicorp/vault/blob/v1.21.4/command/agentproxyshared/auth/auth.go#L99-L101
```go
InvalidToken: make(chan error, 1), // buffered
AuthInProgress: &atomic.Bool{},
```
And the auth handler listens on `ah.InvalidToken` to trigger re-auth:
https://github.com/hashicorp/vault/blob/v1.21.4/command/agentproxyshared/auth/auth.go#L607
```go
case <-ah.InvalidToken:
ah.logger.Info("invalid token found, re-authenticating")
break LifetimeWatcherLoop
```
## Observed Behavior
When running `vault agent` (v1.17.3 tested, also verified in v1.21.4 source) with `auto_auth` + `cache` + `listener`:
1. Revoke the agent's token on the server side
2. Send a request through the agent's cache listener
3. Agent logs: `"proxy received an invalid token error"` (from [handler.go#L77](https://github.com/hashicorp/vault/blob/v1.21.4/command/agentproxyshared/cache/handler.go#L77))
4. But the agent **never** logs `"invalid token found, re-authenticating"` (from [auth.go#L608](https://github.com/hashicorp/vault/blob/v1.21.4/command/agentproxyshared/auth/auth.go#L608))
5. The client request **hangs** until timeout — no HTTP response is returned
The hang occurs because `invalidTokenErrCh` is an unbuffered channel with no reader. The send at [handler.go#L81](https://github.com/hashicorp/vault/blob/v1.21.4/command/agentproxyshared/cache/handler.go#L81) blocks the HTTP handler goroutine, preventing the response from being flushed.
## Expected Behavior
Same as `vault proxy` — after detecting an invalid token, the agent should trigger re-authentication and retry with a fresh token.
## Additional Impact
- Each request that hits this path creates a **blocked goroutine** (~4-8 KB) that is never cleaned up, leading to a small but cumulative resource leak under sustained retry load
- The `authInProgress` variable is also not wired ([`agent.go` L552](https://github.com/hashicorp/vault/blob/v1.21.4/command/agent.go#L552)), so the handler always reads `false` and always attempts the channel send, even during actual re-authentication
- Switching to `vault proxy` is not always feasible — the [Vault Agent Injector for Kubernetes](https://developer.hashicorp.com/vault/docs/deploy/kubernetes/injector) only supports injecting `vault agent`, not `vault proxy` (see [vault-k8s #495](https://github.com/hashicorp/vault-k8s/issues/495))
## Affected Versions
v1.17.0 through v1.21.4, and current `main` branch (`v2.0.2`).
## Possible Fix
Add the two missing assignments in `command/agent.go`, after `ah` is created (around line 600), matching what `command/proxy.go` already does:
```go
authInProgress = ah.AuthInProgress
invalidTokenErrCh = ah.InvalidToken
```
Thank you for your time, and for the great work on the self-healing feature.
Contributor guide
Assessment
This issue has not been assessed yet.