cockroachdb / cockroachdb/cockroach

importer: `ExternalStorage.List` call should be moved out of planning into job execution

Open
#104,725 1 comment 0 reactions 0 assignees View on GitHub
A-disaster-recovery C-bug T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

In https://github.com/cockroachlabs/support/issues/2375 we saw an instance of an import taking several minutes in planning because of the number of objects import had to list during wildcard expansion. We hold a txn open for this duration on the jobs table that was frequently refreshed causing the List call to re-resume. The txn also failed to commit multiple times with https://www.cockroachlabs.com/docs/v22.2/transaction-retry-error-reference.html#abort_reason_client_reject.

Planning should be cheap and not reach out to ExternalStorage. We should move this step into job execution where we are not holding a txn open on the jobs table.

Jira issue: CRDB-28696

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.