cockroachdb / cockroachdb/cockroach

logical,dlq: add DLQ table name overloads for the DLQ builtins

Open
#173,361 8 comments 0 reactions 1 assignee Claimed by @DarrylWong View on GitHub
A-cdc C-enhancement T-cdc
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

The `crdb_dlq_*` builtins currently key off a job ID:

```
information_schema.crdb_dlq_to_json(job_id: int, row_ids: int[])
information_schema.crdb_dlq_to_json(job_id: int, mvcc_timestamp: decimal)
information_schema.crdb_dlq_retry(job_id: int, row_ids: int[])
information_schema.crdb_dlq_retry_at_current_time(job_id: int, row_ids: int[])
```

The job ID is used for two things: loading the job record (for authorization and for the source→destination descriptor ID mapping), and resolving the DLQ database/schema so the entry table `..""` and descriptor table `.."_descriptors"` can be addressed.

That makes the DLQ unreachable once the job is gone. DLQ entries outlive the job that wrote them, and we should support inspecting and retrying them after the job has been cancelled or GC'd.

**Proposal**

Add overloads that take the fully qualified DLQ entry table name instead of a job ID:

```
information_schema.crdb_dlq_to_json(dlq_table: string, row_ids: int[])
information_schema.crdb_dlq_to_json(dlq_table: string, mvcc_timestamp: decimal)
information_schema.crdb_dlq_retry(dlq_table: string, row_ids: int[])
information_schema.crdb_dlq_retry_at_current_time(dlq_table: string, row_ids: int[])
```

The descriptor table is derived from the entry table by appending `_descriptors`, so one argument is enough. The existing job-ID overloads stay and become thin wrappers that resolve the table name and delegate.

**Open questions**

1. *Authorization.* The job-ID path gates on `HasViewAccessToJob` (read) and job control + `REPLICATIONDEST` on the destination tables (retry). With no job, the read path should gate on `SELECT` on the DLQ table itself. Retry still needs `REPLICATIONDEST` on the destination tables — see (2) for how we find them.
2. *Destination table resolution for retry.* `DLQRetry` builds its source→destination descriptor ID map from `LogicalReplicationDetails.ReplicationPairs`. Without a job we need another source for that mapping — either persist it in the DLQ descriptor table, or accept it as an argument. This is the main design decision in the ticket; `crdb_dlq_to_json` has no such dependency and could land first.
3. *Name parsing.* Parse the string as a `tree.TableName` and require it to be fully qualified, rather than resolving against the session search path, so behavior does not depend on `database`/`search_path`.

**Acceptance criteria**

- All four builtins have a string-typed overload registered (including `fixed_oids.go` entries) and documented.
- Viewing and retrying DLQ entries works against a table whose originating job no longer exists.
- Clear errors for a nonexistent, non-DLQ, or non-fully-qualified table name.
- Logic tests covering both overloads and the job-deleted case.

Jira issue: CRDB-66676

-------

Another option could be to:

1. Make the user specify the table prefix, i.e. {table_prefix}_dlq and {table_prefix}_descriptors
2. Only allow accessing via the `{table_prefix}_dlq` and not the job id.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.