cockroachdb / cockroachdb/cockroach

basalt-26.3: backup: TestBackupTenantImportingTable failed

Open
#174,243 6 comments 0 reactions 0 assignees View on GitHub
A-disaster-recovery branch-basalt-26.3 C-test-failure O-robot P-3 T-disaster-recovery
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

backup.TestBackupTenantImportingTable [failed](https://mesolite.cluster.engflow.com/invocations/default/58e2707e-c38d-4b70-beb4-0ce6b35b2170?testReportRun=3&testReportShard=19&testReportAttempt=1#targets-Ly9wa2cvYmFja3VwOmJhY2t1cF90ZXN0) on basalt-26.3 @ [abe45ee7e166185f8b69028bc6d48f2b10e723ce](https://github.com/cockroachdb/cockroach/commits/abe45ee7e166185f8b69028bc6d48f2b10e723ce):

```
| pkg/sql/catalog/descs/getters.go:59
| github.com/cockroachdb/cockroach/pkg/sql.getDBAndRegionEnumDescs
| pkg/sql/region_util.go:1048
| github.com/cockroachdb/cockroach/pkg/sql.GetRegionEnumRepresentations
| pkg/sql/region_util.go:1009
| github.com/cockroachdb/cockroach/pkg/sql.GetLocalityRegionEnumPhysicalRepresentation.func1
| pkg/sql/region_util.go:980
| github.com/cockroachdb/cockroach/pkg/sql.(*InternalDB).DescsTxn.func1
| pkg/sql/internal.go:1986
| github.com/cockroachdb/cockroach/pkg/sql.(*InternalDB).txn.func4
| pkg/sql/internal.go:2089
| github.com/cockroachdb/cockroach/pkg/kv.(*Txn).exec
| pkg/kv/txn.go:1190
| github.com/cockroachdb/cockroach/pkg/kv.runTxn
| pkg/kv/db.go:1169
| github.com/cockroachdb/cockroach/pkg/kv.(*DB).TxnWithAdmissionControl
| pkg/kv/db.go:1129
| github.com/cockroachdb/cockroach/pkg/kv.(*DB).Txn
| pkg/kv/db.go:1101
| github.com/cockroachdb/cockroach/pkg/sql.(*InternalDB).txn
| pkg/sql/internal.go:2076
| github.com/cockroachdb/cockroach/pkg/sql.(*InternalDB).DescsTxn
| pkg/sql/internal.go:1984
| github.com/cockroachdb/cockroach/pkg/sql.GetLocalityRegionEnumPhysicalRepresentation
| pkg/sql/region_util.go:975
| github.com/cockroachdb/cockroach/pkg/server.(*SQLServer).preStart.func3
| pkg/server/server_sql.go:1619
| github.com/cockroachdb/cockroach/pkg/util/startup.RunIdempotentWithRetryEx[...]
| pkg/util/startup/retry.go:137
| github.com/cockroachdb/cockroach/pkg/server.(*SQLServer).preStart
| pkg/server/server_sql.go:1615
| github.com/cockroachdb/cockroach/pkg/server.(*SQLServerWrapper).PreStart
| pkg/server/tenant.go:870
| github.com/cockroachdb/cockroach/pkg/server.(*SQLServerWrapper).Start
| pkg/server/tenant.go:1168
| github.com/cockroachdb/cockroach/pkg/server.(*testServer).StartTenant
| pkg/server/testserver.go:1974
| github.com/cockroachdb/cockroach/pkg/testutils/serverutils.StartTenant
| pkg/testutils/serverutils/test_server_shim.go:460
| github.com/cockroachdb/cockroach/pkg/backup.TestBackupTenantImportingTable
| pkg/backup/backup_tenant_test.go:101
| testing.tRunner
| GOROOT/src/testing/testing.go:2036
| runtime.goexit
| src/runtime/asm_amd64.s:1771
Wraps: (3) database "[1]" does not exist
Error types: (1) *pgerror.withCandidateCode (2) *withstack.withStack (3) *errutil.leafError
panic.go:694: -- test log scope end --
test logs left over in: outputs.zip/logTestBackupTenantImportingTable1004397252
--- FAIL: TestBackupTenantImportingTable (21.96s)
```

Parameters:
- attempt=1
- race=true
- run=3
- shard=19
Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

/cc @cockroachlabs/disaster-recovery

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestBackupTenantImportingTable.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-67547

Contributor guide

Open the contributing guide

Research direction

Start with pkg/backup/backup_tenant_test.go and run TestBackupTenantImportingTable with the reported race, run, and shard parameters. Trace the tenant startup failure through pkg/server/server_sql.go, pkg/sql/region_util.go, and pkg/sql/catalog/descs/getters.go, using the linked test report and leftover test logs. Done means the test completes successfully without the database-does-not-exist error.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.