`database.{NAME}.*` telemetry is keyed by plugin type, not by mount or connection name (docs inaccuracy + request for per-mount labels)
- Dominant language
- Go
- Stars
- 36.3k
- Forks
- 4.8k
- PR merge metrics
- PR metrics pending
Description
## Summary
Two related things:
1. **Docs bug** — [Database telemetry metrics](https://developer.hashicorp.com/vault/docs/internals/telemetry/metrics/database) describes the `{NAME}` in `database.{NAME}.*` as the named secrets engine. It is actually the **database plugin type string**, and for PostgreSQL it isn't even `postgresql` — it's `pgx`.
2. **Feature request** — there is currently no way to break database secrets engine telemetry down per mount or per configured connection. I'd like that.
## The documentation is inaccurate
The page currently states that enabling a PostgreSQL secrets engine called `postgresql-prod` produces `database.postgresql-prod.CreateUser.error`, and describes `database.{NAME}.Close` as the time to close "the database secrets engine {NAME}".
Both are wrong. `{NAME}` is neither the mount path nor the connection name, and the PostgreSQL plugin's type string is `pgx`. The real metric is:
```
database.pgx.CreateUser.error
```
### Where it comes from
`{NAME}` is `typeStr`, obtained from the plugin itself in `sdk/database/dbplugin/v5/plugin_factory.go` (~L87):
```go
typeStr, err := db.Type()
...
// Wrap with metrics middleware
db = &databaseMetricsMiddleware{
next: db,
typeStr: typeStr,
}
```
`sdk/database/dbplugin/v5/middleware.go` emits it directly, with no labels:
```go
metrics.MeasureSince([]string{"database", mw.typeStr, "Close"}, now)
```
The connection name never reaches the factory. In `builtin/logical/database/backend_ce.go` the connection name (`name`) is in scope and used for the connection cache, but only `config.PluginName` is forwarded:
```go
dbw, err := newDatabaseWrapper(ctx, config.PluginName, pluginVersion, b.System(), b.logger)
```
### Reproduction
I confirmed this by running the engine with an in-memory telemetry sink: mounted the database secrets engine, registered the `postgresql-database-plugin`, and created a connection named `zzz-my-distinctive-connection-name`. Every metric emitted:
```
counter database.Close
counter database.Initialize
counter database.pgx.Close
counter database.pgx.Initialize
sample database.Close
sample database.Initialize
sample database.pgx.Close
sample database.pgx.Initialize
key contains connection name "zzz-my-distinctive-connection-name" : false
key contains "pgx" : true
key contains "postgresql-database-plugin" : false
```
Vault's own debug log in that run:
```
[DEBUG] got database plugin instance: type=pgx
[DEBUG] created database object: name=zzz-my-distinctive-connection-name plugin_name=postgresql-database-plugin
```
The connection name is known to Vault at that exact moment and still does not appear.
### Actual `{NAME}` values
Verified by executing each plugin's `Type()`:
| Plugin | `{NAME}` |
| --- | --- |
| `postgresql-database-plugin` | `pgx` |
| `mysql-database-plugin` | `mysql` |
| `mssql-database-plugin` | `mssql` |
| `mongodb-database-plugin` | `mongodb` |
| `cassandra-database-plugin` | `cassandra` |
Read from source constants but not executed: `hana` → `hdb`, `influxdb` → `influxdb`, `redshift` → `redshift`. The redshift plugin's constant carries a comment saying it exists specifically for how the plugin appears in metrics middleware.
### Practical consequence
Every PostgreSQL connection, across every mount and every namespace, aggregates into a single `database.pgx.*` series. Operators reading the current docs will build dashboards expecting per-mount series that will never exist.
## Feature request — per-mount and per-connection labels
### Problem
I want to answer questions like "which mount is generating credential churn" and "is `orders-primary` slower than `reporting-replica`". Today that's impossible: the only dimension available is plugin type.
### Proposed solution
Attach `mount_point` and `connection_name` as **labels**, gated behind a new telemetry option, off by default:
```hcl
telemetry {
add_mount_point_database_metrics = true
}
```
Resulting emission:
```
# default (option absent or false) — unchanged from today
database.Initialize
database.pgx.Initialize
# with the option enabled
database.Initialize;mount_point=db-prod/;connection_name=orders-primary
database.pgx.Initialize;mount_point=db-prod/;connection_name=orders-primary
```
### Why labels rather than name segments
`vault.rollback.attempt.{MOUNT_POINT}` and `vault.route.rollback.{MOUNT_POINT}` were deliberately replaced with unsuffixed names because mount points in metric *names* explode Prometheus series counts, with `add_mount_point_rollback_metrics` added as the opt-in escape hatch. Repeating that pattern here would recreate the same problem and break existing queries. Labels leave metric names untouched, so current dashboards keep working and aggregate exactly as before.
### Why opt-in
Series count scales with mounts × connections × operations. Both closest precedents (`add_mount_point_rollback_metrics`, `add_lease_metrics_namespace_labels`) are booleans defaulting to false, so this follows the established convention.
Contributor guide
Research direction
Start with the database telemetry metrics documentation, then read sdk/database/dbplugin/v5/plugin_factory.go and middleware.go to trace typeStr into emitted metrics. Compare that flow with builtin/logical/database/backend_ce.go, where the connection name is in scope, and verify that the documentation matches the observed plugin-type metrics while default telemetry remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, documentation, observability-sre
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100