cockroachdb / cockroachdb/cockroach
schema: add view backfill job progress for observability
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
There is a long running schema change job that is stuck here ([context](https://github.com/cockroachlabs/support/issues/2412#issuecomment-1609817110)):
```
1 @ 0x4907d6 0x45a91b 0x45a458 0x25760da 0x2575ec3 0x2575a74 0x1188585 0x1187ce4 0x4c2401
# labels: {"job":"SCHEMA CHANGE id=877503506960875522", "n":"4"}
# 0x25760d9 github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*bulkRowWriter).ingestLoop.func1+0x99 github.com/cockroachdb/cockroach/pkg/sql/rowexec/bulk_row_writer.go:160
# 0x2575ec2 github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*bulkRowWriter).ingestLoop+0x2c2 github.com/cockroachdb/cockroach/pkg/sql/rowexec/bulk_row_writer.go:175
# 0x2575a73 github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*bulkRowWriter).work.func1+0x33 github.com/cockroachdb/cockroach/pkg/sql/rowexec/bulk_row_writer.go:118
# 0x1188584 github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1+0x24 github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168
# 0x1187ce3 golang.org/x/sync/errgroup.(*Group).Go.func1+0x63 golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:74
```
Its progress appears as 0% even though it has been running for a while - the job should update its progress for observability with
1) Number of rows backfilled
2) A delay warning if it is not making progress after X amount of time
Jira issue: CRDB-29133
Epic CRDB-42485
Contributor guide
Assessment
This issue has not been assessed yet.