cockroachdb / cockroachdb/cockroach
jobs: add markers to categorize user created jobs and background service jobs
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
As suggested by @dt we should add a few more bits of information to be able to categorize jobs into:
- System task/background job
- User-created job
Within the `System task` category we should further subdivide into:
- Forever running job
- One-time task that runs and completes
The primary motivation for this categorization is to improve the UX around how users can track currently running jobs in the DB console. Today, the console has an `ALL` jobs tab, and then each job type has a separate tab. Confusingly, the `ALL` jobs table does not show _all_ jobs for example all the automatic jobs such as key viz, create stats, span config reconciliation etc. If we were to categorize jobs then we could split the `System tasks` from the `User-created` jobs and make it easier for customers to identify all the processes that are currently running in their cluster.
Furthermore, forever running background services should probably not be incrementing and decrementing job-related metrics such as `jobs_running_non_idle`. This metric is currently used by serverless to decide whether or not to scale down a SQL pod. In the future, this is going to be used to annotate Prometheus graphs with information about processes that are doing work in the cluster. It would be useful to separate jobs from services by introducing another `system_service_running_non_idle`. This can be toggled when the background services are actually doing work rather than just idling waiting for their next run.
Jira issue: CRDB-27113
Contributor guide
Research direction
Start by reviewing the DB console's ALL jobs tab and the existing jobs_running_non_idle metric behavior, including automatic jobs such as key viz, create stats, and span config reconciliation. Define how user-created jobs, forever-running services, and one-time system tasks are categorized, and verify that job and service activity can be tracked separately as described in CRDB-27113.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100