erigontech / erigontech/erigon
cl: no metric for local-vs-relay block production source
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.5k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 455
Description
Caplin decides every proposal between the local EL payload and an MEV relay bid, but that decision is only visible in logs. There is no metric, so an operator cannot chart or alert on how their proposals split between the two.
I enumerated every metric registered in the tree (443 names on `main`). Nothing counts proposals by source. The closest is `block_producer_delay{type="production"}` (`execution/metrics/block.go:36`), whose Prometheus `_count` gives local EL payload *builds* — but it increments even when the relay bid wins, so it cannot express the split.
## Why this matters for validator nodes
Choosing local execution when a relay bid was several times larger costs the operator real money, silently. #14883 is exactly that: `builderValue` was ~19x `execValue`, `boostFactor=0` made Caplin pick local anyway, and the operator only discovered it by reading logs after the block was already proposed.
Today the evidence exists only as log lines:
- `cl/beacon/handler/block_production.go:889` — `Check mev bid` with `useLocalExec`, `execValue`, `builderValue`, `boostFactor`
- `cl/beacon/handler/block_production.go:672` — `BlockProduction: Block produced` with `blinded`
- `cl/beacon/handler/block_production.go:1676` — `Successfully submitted blinded block`
Proposals are rare, so a regression here can sit unnoticed for weeks. A counter makes it a dashboard panel and an alert instead of an archaeology exercise.
## Proposed metric
One counter, two label values, incremented at the `useLocalExec` branch:
```
block_production_source{source="local"}
block_production_source{source="builder"}
```
Suggested second counter, because the two genuinely diverge: choosing the relay is not the same as the relay block landing. If `SubmitBlindedBlocks` fails at `block_production.go:1670` the proposer misses the slot entirely — the worst outcome and the one most worth alerting on.
```
block_production_blinded_publish{result="ok"|"error"}
```
Both belong on the Caplin side (`cl/monitor/metrics.go`), since the EL never learns which payload was used.
Optional, cheap while touching this code: a gauge for the last observed `execValue` / `builderValue` pair would show *how much* was left on the table, not just which side won. #14883 would have been obvious from that alone.
## Reuse beyond Caplin
Worth designing with these in mind rather than solving now:
- **External CL + mev-boost.** The gap does not apply — mev-boost and the CL export their own proposal metrics, and Erigon's EL genuinely cannot tell whether its payload was used. Nothing to add.
- **EL-side attribution (needs design).** Erigon already keeps the last built block in `LatestBlockBuiltStore` (`execution/builder/latest_block_built.go`) for `eth_getBlockByNumber("pending")`. When a block arrives whose `parentHash` and `timestamp` both match a payload we built, that is the same slot, so a differing block hash means our proposal went out through a relay. That would give a local-vs-external signal from the EL alone and work under any CL. It needs care around missed slots and reorgs before it can be trusted, so it should not block the Caplin counter above.
- **Naming.** If EL-side attribution lands later, both sources should share the `source="local"|"builder"` label vocabulary so one dashboard works either way.
## Scope
`cl/monitor/metrics.go` plus the increment sites in `cl/beacon/handler/block_production.go`. No behaviour change. Ships on `/debug/metrics/prometheus` with `--metrics`, so it needs a panel in `dashboards/erigon_custom_metrics/` to be useful.
Contributor guide
Assessment
This issue has not been assessed yet.