cockroachdb / cockroachdb/cockroach
jobs: eliminate scans of the job_info table
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
In 23.1 we introduced the system.job_info table and moved the progress and payload columns out of the system.jobs table into the info table. This helped us move per-job writes out of the table that the job system scans, i.e. out of system.jobs table. As part of the migration however we had to introduce a join that scans both the system.jobs and system.job_info table to power some of the other queries in crdb egs: SHOW JOBS. The payload and progress rows in the job_info table are expected to be written and re-written many times and so the intents on these rows could interfere with scans of this table. Furthermore, no tool should interfere with the _execution_ of jobs as this could have a destabilizing impact.
This issue tracks the work needed to get non-job execution queries out of the system.job_info table and into columns on the sysetm.jobs table. One example is the `FinishedMicros` field that is currently in the jobs payload which means every `SHOW JOBS` query (think DB Conosle jobs page) will have to unmarshal the payload of each job and scan the job_info table to do so. Moving this field to a column in the system.jobs table means that scan no longer occurs.
Jira issue: CRDB-32486
Epic CRDB-32494
gz#22845
Contributor guide
Assessment
This issue has not been assessed yet.