pg_table_is_visible slower in CockroachDB 23.1

**Describe the problem**

The Django test suite takes 60-65 minutes with CockraochDB 23.1.x compared to 40-45 minutes with CockroachDB 22.2.x. Some attempts at improving performance were https://github.com/cockroachdb/cockroach/issues/93955 and https://github.com/cockroachdb/cockroach/issues/100871 but the issue remains.

**To Reproduce**

Running the `inspectdb` Django test app as an example (`./runtests.py inspectdb --settings=test_roach`), the run time went from 31 seconds in 22.2.x to 40 seconds with 23.1.x. I turned on slow query logging to 1ms to log all queries. Then I wrote a script to parse the logs and group similar queries. 8 seconds of the 9-second performance difference are attributed to five groups of queries that use `pg_table_is_visible`.

Queries that include `pg_table_is_visible` when running `inspectdb` test app:
```
Statement | 23.1.x seconds | 22.1.x seconds | Query Count |
SELECT ‹c›.‹conname› | 20.55 | 14.24 | 108 |
SELECT ‹indexname›, | 12.66 | 11.80 | 108 |
SELECT ‹a›.‹attname› | 3.65 | 1.47 | 47 |
SELECT ‹c›.‹relname› | 2.15 | 0.94 | 35 |
SELECT ‹a1›.‹attname | 1.94 | 0.30 | 47 |
```

Full queries:
```sql
SELECT ‹c›.‹relname›, CASE WHEN ‹c›.‹relispartition› THEN ‹'p'› WHEN ‹c›.‹relkind› IN (‹'m'›, ‹'v'›) THEN ‹'v'› ELSE ‹'t'› END, ‹''› FROM pg_catalog.pg_class AS ‹c› LEFT JOIN pg_catalog.pg_namespace AS ‹n› ON ‹n›.‹oid› = ‹c›.‹relnamespace› WHERE ((‹c›.‹relkind› IN (‹'f'›, ‹'m'›, ‹'p'›, ‹'r'›, ‹'v'›)) AND (‹n›.‹nspname› NOT IN (‹‹'pg_catalog'››, ‹‹'pg_toast'››))) AND pg_table_is_visible(‹c›.‹oid›)
SELECT ‹a1›.‹attname›, ‹c2›.‹relname›, ‹a2›.‹attname› FROM ‹""›.‹""›.‹pg_constraint› AS ‹con› LEFT JOIN ‹""›.‹""›.‹pg_class› AS ‹c1› ON ‹con›.‹conrelid› = ‹c1›.‹oid› LEFT JOIN ‹""›.‹""›.‹pg_class› AS ‹c2› ON ‹con›.‹confrelid› = ‹c2›.‹oid› LEFT JOIN ‹""›.‹""›.‹pg_attribute› AS ‹a1› ON (‹c1›.‹oid› = ‹a1›.‹attrelid›) AND (‹a1›.‹attnum› = ‹con›.‹conkey›[‹1›]) LEFT JOIN ‹""›.‹""›.‹pg_attribute› AS ‹a2› ON (‹c2›.‹oid› = ‹a2›.‹attrelid›) AND (‹a2›.‹attnum› = ‹con›.‹confkey›[‹1›]) WHERE (((‹c1›.‹relname› = ‹'inspectdb_charfielddbcollation'›) AND (‹con›.‹contype› = ‹'f'›)) AND (‹c1›.‹relnamespace› = ‹c2›.‹relnamespace›)) AND pg_table_is_visible(‹c1›.‹oid›)
SELECT ‹c›.‹conname›, ARRAY (SELECT ‹attname› FROM ROWS FROM (unnest(‹c›.‹conkey›)) WITH ORDINALITY AS ‹cols› (‹colid›, ‹arridx›) JOIN ‹""›.‹""›.‹pg_attribute› AS ‹ca› ON ‹cols›.‹colid› = ‹ca›.‹attnum› WHERE ‹ca›.‹attrelid› = ‹c›.‹conrelid› ORDER BY ‹cols›.‹arridx›), ‹c›.‹contype›, (SELECT (‹fkc›.‹relname› || ‹'.'›) || ‹fka›.‹attname› FROM ‹""›.‹""›.‹pg_attribute› AS ‹fka› JOIN ‹""›.‹""›.‹pg_class› AS ‹fkc› ON ‹fka›.‹attrelid› = ‹fkc›.‹oid› WHERE (‹fka›.‹attrelid› = ‹c›.‹confrelid›) AND (‹fka›.‹attnum› = ‹c›.‹confkey›[‹1›])), ‹cl›.‹reloptions› FROM ‹""›.‹""›.‹pg_constraint› AS ‹c› JOIN ‹""›.‹""›.‹pg_class› AS ‹cl› ON ‹c›.‹conrelid› = ‹cl›.‹oid› WHERE (‹cl›.‹relname› = ‹'inspectdb_charfielddbcollation'›) AND pg_table_is_visible(‹cl›.‹oid›)
SELECT ‹indexname›, array_agg(‹attname› ORDER BY ‹arridx›), ‹indisunique›, ‹indisprimary›, array_agg(‹ordering› ORDER BY ‹arridx›), ‹amname›, ‹exprdef›, ‹s2›.‹attoptions› FROM (SELECT ‹c2›.‹relname› AS ‹indexname›, ‹idx›.*, ‹attr›.‹attname›, ‹am›.‹amname›, CASE WHEN ‹idx›.‹indexprs› IS NOT NULL THEN pg_get_indexdef(‹idx›.‹indexrelid›) END AS ‹exprdef›, CASE ‹am›.‹amname› WHEN ‹'prefix'› THEN CASE (‹option› & ‹1›) WHEN ‹1› THEN ‹'DESC'› ELSE ‹'ASC'› END END AS ‹ordering›, ‹c2›.‹reloptions› AS ‹attoptions› FROM (SELECT * FROM ‹""›.‹""›.‹pg_index› AS ‹i›, ROWS FROM (unnest(‹i›.‹indkey›, ‹i›.‹indoption›)) WITH ORDINALITY AS ‹koi› (‹key›, ‹option›, ‹arridx›)) AS ‹idx› LEFT JOIN ‹""›.‹""›.‹pg_class› AS ‹c› ON ‹idx›.‹indrelid› = ‹c›.‹oid› LEFT JOIN ‹""›.‹""›.‹pg_class› AS ‹c2› ON ‹idx›.‹indexrelid› = ‹c2›.‹oid› LEFT JOIN ‹""›.‹""›.‹pg_am› AS ‹am› ON ‹c2›.‹relam› = ‹am›.‹oid› LEFT JOIN ‹""›.‹""›.‹pg_attribute› AS ‹attr› ON (‹attr›.‹attrelid› = ‹c›.‹oid›) AND (‹attr›.‹attnum› = ‹idx›.‹key›) WHERE (‹c›.‹relname› = ‹'inspectdb_charfielddbcollation'›) AND pg_table_is_visible(‹c›.‹oid›)) AS ‹s2› GROUP BY ‹indexname›, ‹indisunique›, ‹indisprimary›, ‹amname›, ‹exprdef›, ‹attoptions›
SELECT ‹a›.‹attname› AS ‹column_name›, NOT (‹a›.‹attnotnull› OR ((‹t›.‹typtype› = ‹'d'›) AND ‹t›.‹typnotnull›)) AS ‹is_nullable›, pg_get_expr(‹ad›.‹adbin›, ‹ad›.‹adrelid›) AS ‹column_default›, CASE WHEN ‹collname› = ‹'default'› THEN ‹NULL› ELSE ‹collname› END AS ‹collation›, ‹a›.‹attidentity› != ‹''› AS ‹is_autofield›, col_description(‹a›.‹attrelid›, ‹a›.‹attnum›) AS ‹column_comment› FROM ‹""›.‹""›.‹pg_attribute› AS ‹a› LEFT JOIN ‹""›.‹""›.‹pg_attrdef› AS ‹ad› ON (‹a›.‹attrelid› = ‹ad›.‹adrelid›) AND (‹a›.‹attnum› = ‹ad›.‹adnum›) LEFT JOIN ‹""›.‹""›.‹pg_collation› AS ‹co› ON ‹a›.‹attcollation› = ‹co›.‹oid› JOIN ‹""›.‹""›.‹pg_type› AS ‹t› ON ‹a›.‹atttypid› = ‹t›.‹oid› JOIN ‹""›.‹""›.‹pg_class› AS ‹c› ON ‹a›.‹attrelid› = ‹c›.‹oid› JOIN ‹""›.‹""›.‹pg_namespace› AS ‹n› ON ‹c›.‹relnamespace› = ‹n›.‹oid› WHERE (((‹c›.‹relkind› IN (‹'f'›, ‹'m'›, ‹'p'›, ‹'r'›, ‹'v'›)) AND (‹c›.‹relname› = ‹'inspectdb_charfielddbcollation'›)) AND (‹n›.‹nspname› NOT IN (‹‹'pg_catalog'››, ‹‹'pg_toast'››))) AND pg_table_is_visible(‹c›.‹oid›)
```

**Environment:**
- CockroachDB version v23.2.0-alpha built 2023/07/25 and v22.2.12

Jira issue: CRDB-30428