cockroachdb / cockroachdb/cockroach
streamingccl: add even more LDR metrics to DB console
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
After #125320, a @dt @ajstorm wished for more metrics. We should add these when we have dev time or wish we had them during a debugging session:
Source side
- [ ] Row Updates Sent i.e. num kvs that went into batches that eventStream sent (single plot of sum across nodes)
- [ ] Megabytes Sent i.e. sum size of KVs that went out of eventStream (one plot per-node)
- [ ] Emission Queue Delay i.e. how long eventStream waits for Next() calls (two plots for p50+p99 per node)
Destination side
- [ ] Application Conflicts Placeholder / TODO.
- [ ] Application Failures / Updates send to DLQ Placeholder / TODO.
- [ ] Batches Applied ie. number of update/insert/delete queries run by the consumer (one plot per node)
- [ ] replanning events - i.e. the amount of times we had to replan the job
Jira issue: CRDB-39500
Contributor guide
Research direction
Start by reviewing issue #125320 and the streamingccl DB console work it introduced. Compare the source-side and destination-side metric checklist here, including the placeholder items, and consider the work done when the requested metrics are exposed in the console with the listed aggregation and per-node or percentile views.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100