apache / apache/shardingsphere

filters out user tables with "$" in names as system tables

Open
#36,462 7 comments 0 reactions 0 assignees View on GitHub
in: metadata
Dominant language
Java
Stars
20.8k
Forks
6.9k
Avg merge
9h 55m
Merged PRs (30d)
311

Description

## Bug Report

**For English only**, other languages will not accept.

Before report a bug, make sure you have:

- Searched open and closed [GitHub issues](https://github.com/apache/shardingsphere/issues).
- Read documentation: [ShardingSphere Doc](https://shardingsphere.apache.org/document/current/en/overview).

Please pay attention on issues you submitted, because we maybe need more details.
If no response anymore and we cannot reproduce it on current information, we will **close it**.

Please answer these questions before submitting your issue. Thanks!

### Which version of ShardingSphere did you use?

5.5.2

### Which project did you use? ShardingSphere-JDBC or ShardingSphere-Proxy?

ShardingSphere-Proxy

----------------------------

Hi everyone.

While working with Proxy, I ran into an issue: tables with a `$` in their name do not work correctly — they are not loaded into the schema during metadata loading, and any queries to them fail with an error (`Table or view 'R$TABLE' does not exist`). I reproduced this problem on both `Firebird` and `PostgreSQL`.

Looking at the code, such tables are filtered out during metadata loading in [SchemaMetaDataLoader](https://github.com/apache/shardingsphere/blob/789ee8ca36981737d3a9d1e98e08b2ec1806467e/database/connector/core/src/main/java/org/apache/shardingsphere/database/connector/core/metadata/data/loader/type/SchemaMetaDataLoader.java#L41) by the following rule:

```Java
private boolean isSystemTable(final String table) {
return table.contains("/") || table.contains("##") || table.contains("$");
}
```
This filter is used in the table loading method:

```Java
private Collection loadTableNames(final Connection connection, final String schemaName, final Collection excludedTables) throws SQLException {
Collection result = new CaseInsensitiveSet<>();
String[] tableTypes = new String[]{TABLE_TYPE, PARTITIONED_TABLE_TYPE, VIEW_TYPE, SYSTEM_TABLE_TYPE, SYSTEM_VIEW_TYPE};
try (ResultSet resultSet = connection.getMetaData().getTables(connection.getCatalog(), schemaName, null, tableTypes)) {
while (resultSet.next()) {
String table = resultSet.getString(TABLE_NAME);
if (!isSystemTable(table) && !excludedTables.contains(table)) {
result.add(table);
}
}
}
return result;
}
```
This means that any table whose name contains `$` is automatically treated as “system” and excluded from the user schema. The problem is that in both `PostgreSQL` and `Firebird`, `$` is perfectly valid in user identifiers and does not imply a system object. As a result, valid user tables with `$` are filtered out at Proxy startup and become unusable.

At the same time, system tables for each database are already explicitly defined in `system/schema (YAML)`. This mechanism allows Proxy to recognize which tables are truly system ones and handle them separately. Given that, it’s not clear why we still need a global heuristic like `contains("$")`. Wouldn’t it be more reliable to check against the system tables list for the current `databaseType` (e.g., for `Firebird`: `RDB$*`) rather than applying generic character-based rules?

I understand that this rule was originally introduced to prevent loading system tables that are not yet fully implemented, but it would be better if it were refined so that it does not also break ordinary user tables.

Right now the global `$` rule creates a contradiction: system tables are already covered by `system/schema`, but ordinary user tables containing `$` are also excluded and lost. Should this behavior be reconsidered so that user tables with `$` remain available after Proxy restarts? Otherwise, it becomes impossible to work with non-system tables whose names happen to include $.

P.S. I also noticed the [DriverQuerySystemCatalogOption](https://github.com/apache/shardingsphere/blob/789ee8ca36981737d3a9d1e98e08b2ec1806467e/database/connector/core/src/main/java/org/apache/shardingsphere/database/connector/core/metadata/database/metadata/option/table/DialectDriverQuerySystemCatalogOption.java#L23) interface and its `isSystemTable(...)` method. At first I thought I could override it to define system tables in a dialect-specific way (as done for openGauss). But it turns out this interface is not used during metadata loading in `SchemaMetaDataLoader`; instead, it is used later during query execution (SQL federation) to recognize system catalog queries and choose a special execution path (e.g., in-memory results / avoiding backend scans). That makes sense for offloading Proxy and handling system queries, but at startup the `contains("$")` rule still applies and filters out such tables.

Contributor guide

Open the contributing guide

Research direction

Start with SchemaMetaDataLoader.java, especially isSystemTable and loadTableNames, then review the system/schema YAML definitions mentioned in the issue. Reproduce metadata loading with user tables containing "$" on PostgreSQL or Firebird and verify that valid user tables remain available while system tables are still handled correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, postgresql
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.