cockroachdb / cockroachdb/cockroach

table_metadata_updater.go contains inefficient query

Closed
#158,488 5 comments 0 reactions 1 assignee Assigned to @alyshanjahani-crl View on GitHub
A-many-descriptors branch-master C-bug O-support P-3 T-observability
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

This query can be very inefficient when both the `system.table_metadata` and the `system.namespace` tables are large: https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/tablemetadatacache/table_metadata_updater.go#L155

```
DELETE FROM system.table_metadata
WHERE table_id IN (
SELECT table_id
FROM system.table_metadata
WHERE table_id NOT IN (
SELECT id FROM system.namespace
)
LIMIT $1
)
RETURNING table_id
```

We had a cluster which initially contained several thousand tables and thousands of schemas created through an automated process. The updating of the `system.table_metadata` table took a long time and so the cluster owner decided to drop the majority of their tables. However, the database still contained around 80,000 schema objects so the namespace table was still very large. The inner `SELECT table_id` took nearly 15 minutes to return a batch of just 20 rows to be deleted. With a lot of recently dropped objects (in the thousands), this step was taking forever. In the end, I had to manually remove rows from the `system.table_metadata` table.

**Suggestion**

We replace the above query with the following:

```
DELETE FROM system.table_metadata
WHERE table_id NOT IN (
SELECT id FROM system.descriptor
)
LIMIT $1
RETURNING table_id
```

It does mean we wait a little longer to remove the row (we wait for the table to be GC'd rather than from the point it is dropped), but the above took just a couple of seconds.

Jira issue: CRDB-57333

Epic CRDB-55226

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.