cockroachdb / cockroachdb/cockroach

jobs: eliminate scans of the job_info table

Open
#112,547 0 comments 0 reactions 0 assignees View on GitHub
A-jobs C-enhancement O-postmortem P-3 T-jobs
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

In 23.1 we introduced the system.job_info table and moved the progress and payload columns out of the system.jobs table into the info table. This helped us move per-job writes out of the table that the job system scans, i.e. out of system.jobs table. As part of the migration however we had to introduce a join that scans both the system.jobs and system.job_info table to power some of the other queries in crdb egs: SHOW JOBS. The payload and progress rows in the job_info table are expected to be written and re-written many times and so the intents on these rows could interfere with scans of this table. Furthermore, no tool should interfere with the _execution_ of jobs as this could have a destabilizing impact.

This issue tracks the work needed to get non-job execution queries out of the system.job_info table and into columns on the sysetm.jobs table. One example is the `FinishedMicros` field that is currently in the jobs payload which means every `SHOW JOBS` query (think DB Conosle jobs page) will have to unmarshal the payload of each job and scan the job_info table to do so. Moving this field to a column in the system.jobs table means that scan no longer occurs.

Jira issue: CRDB-32486

Epic CRDB-32494

gz#22845

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.