cockroachdb / cockroachdb/cockroach

schema: add view backfill job progress for observability

Open
#105,634 1 comment 0 reactions 0 assignees View on GitHub
A-schema-changes C-bug T-sql-foundations
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

There is a long running schema change job that is stuck here ([context](https://github.com/cockroachlabs/support/issues/2412#issuecomment-1609817110)):
```
1 @ 0x4907d6 0x45a91b 0x45a458 0x25760da 0x2575ec3 0x2575a74 0x1188585 0x1187ce4 0x4c2401
# labels: {"job":"SCHEMA CHANGE id=877503506960875522", "n":"4"}
# 0x25760d9 github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*bulkRowWriter).ingestLoop.func1+0x99 github.com/cockroachdb/cockroach/pkg/sql/rowexec/bulk_row_writer.go:160
# 0x2575ec2 github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*bulkRowWriter).ingestLoop+0x2c2 github.com/cockroachdb/cockroach/pkg/sql/rowexec/bulk_row_writer.go:175
# 0x2575a73 github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*bulkRowWriter).work.func1+0x33 github.com/cockroachdb/cockroach/pkg/sql/rowexec/bulk_row_writer.go:118
# 0x1188584 github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1+0x24 github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168
# 0x1187ce3 golang.org/x/sync/errgroup.(*Group).Go.func1+0x63 golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:74
```

Its progress appears as 0% even though it has been running for a while - the job should update its progress for observability with
1) Number of rows backfilled
2) A delay warning if it is not making progress after X amount of time

Jira issue: CRDB-29133

Epic CRDB-42485

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.