apache / apache/shardingsphere

Subquery with non-binding sharding tables causes full-route broadcast

Open
#38,456 1 comment 0 reactions 0 assignees View on GitHub
feature: sharding type: bug
Dominant language
Java
Stars
20.8k
Forks
6.9k
Avg merge
11h 35m
Merged PRs (30d)
326

Description

## Bug Report

### Which version of ShardingSphere did you use?

5.4.1 (also confirmed on master branch)

### Which project did you use? ShardingSphere-JDBC or ShardingSphere-Proxy?

ShardingSphere-JDBC

### Expected behavior

The following subquery SQL with two non-binding sharding tables should generate 1 actual SQL, routed to a single datasource and single table shard.

### Actual behavior

512 actual SQLs are generated (4 datasources × 128 table shards), causing a full-route broadcast. The query fails with:

```
PSQLException: ERROR: relation "um_merchant_user_tag_32" does not exist
```

Note: Rewriting the same query as a JOIN produces the correct result (1 SQL). The issue is specific to subqueries.

### Reason analyze

Through debugging, I found the issue originates from a difference in how `ShardingConditionEngine` organizes conditions for JOIN vs subquery.

**JOIN version** — conditions are merged into a single `ShardingCondition` with 2 values:
```
Condition #1: values = [um_merchant_user_tag.merchant_id = 1074307, um_tag_user.merchant_id = 1074307]
```

**Subquery version** — conditions are split into 2 separate `ShardingCondition` entries:
```
Condition #1: values = [um_merchant_user_tag.merchant_id = 1074307]
Condition #2: values = [um_tag_user.merchant_id = 1074307]
```

In `ShardingStandardRouteEngine.routeByShardingConditionsWithCondition()`, the code iterates over all conditions and accumulates results via `result.addAll(dataNodes)`:

```java
for (ShardingCondition each : shardingConditions.getConditions()) {
Collection dataNodes = route0(shardingTable,
databaseShardingStrategy, getShardingValuesFromShardingConditions(..., each),
tableShardingStrategy, getShardingValuesFromShardingConditions(..., each));
result.addAll(dataNodes);
originalDataNodes.add(dataNodes);
}
```

When routing `um_merchant_user_tag`:

1. **Condition #1** (`um_merchant_user_tag.merchant_id`) — tableName matches → precise routing → **1 DataNode**
2. **Condition #2** (`um_tag_user.merchant_id`) — tableName does not match, not a binding table → `getShardingValuesFromShardingConditions` returns empty → `route0` treats empty as "no condition" → full broadcast → **512 DataNodes**
3. `result.addAll` accumulates both → **513 DataNodes**

The same issue occurs symmetrically when routing `um_tag_user`.

### Possible fix

Skip conditions that are irrelevant to the current routing table before entering `route0()`. I have verified this approach passes all existing tests and the new reproduction test case. The same pattern exists in `routeByMixedConditionsWithCondition()`.

Would like to confirm with maintainers whether this direction is acceptable.

```java
private Collection routeByShardingConditionsWithCondition(final ShardingRule shardingRule, final ShardingTable shardingTable,
final ShardingStrategy databaseShardingStrategy, final ShardingStrategy tableShardingStrategy) {
Collection result = new LinkedList<>();
for (ShardingCondition each : shardingConditions.getConditions()) {
if (!isConditionRelevantToTable(shardingRule, each)) {
continue;
}
Collection dataNodes = route0(shardingTable,
databaseShardingStrategy, getShardingValuesFromShardingConditions(shardingRule, databaseShardingStrategy.getShardingColumns(), each),
tableShardingStrategy, getShardingValuesFromShardingConditions(shardingRule, tableShardingStrategy.getShardingColumns(), each));
result.addAll(dataNodes);
originalDataNodes.add(dataNodes);
}
if (result.isEmpty()) {
return route0(shardingTable, databaseShardingStrategy, Collections.emptyList(), tableShardingStrategy, Collections.emptyList());
}
return result;
}

private boolean isConditionRelevantToTable(final ShardingRule shardingRule, final ShardingCondition condition) {
for (ShardingConditionValue value : condition.getValues()) {
if (logicTableName.equalsIgnoreCase(value.getTableName())) {
return true;
}
Optional bindingTableRule = shardingRule.findBindingTableRule(value.getTableName());
if (bindingTableRule.isPresent() && bindingTableRule.get().hasLogicTable(logicTableName)) {
return true;
}
}
return false;
}
```

### Steps to reproduce the behavior

**Database:** PostgreSQL, JDK 17

**Sharding configuration:**

Two sharding tables sharing the same database sharding algorithm (`merchant_id`), but with asymmetric table sharding strategies. They cannot be configured as binding tables due to the strategy mismatch.

| Table | Database Strategy | Table Strategy | Actual Nodes |
|-------|------------------|----------------|--------------|
| `um_tag_user` | standard (merchant_id) | none (database-only) | ds0~ds3, 1 table per ds |
| `um_merchant_user_tag` | standard (merchant_id) | standard (merchant_id, 128 tables) | ds0~ds3, 128 tables per ds |

For `merchant_id = 1074307`, both tables route to the same datasource, and `um_merchant_user_tag` routes to table shard `_3`.

**SQL to execute:**

```sql
SELECT COUNT(t.id) AS num
FROM um_tag_user t
WHERE t.id IN (
SELECT unnest(u.user_tags)
FROM um_merchant_user_tag u
WHERE u.merchant_id = ?
)
AND t.merchant_id = ?
```

Parameters: `[1074307, 1074307]`

### Example codes for reproduce this issue

Minimal unit test (uses 2 datasources + 4 table shards to reproduce the same issue at smaller scale):

SubqueryNonBindingTableRouteTest.java

```java
package org.apache.shardingsphere.sharding.route.engine.type.standard;

import org.apache.shardingsphere.infra.binder.context.statement.SQLStatementContext;
import org.apache.shardingsphere.infra.binder.engine.SQLBindEngine;
import org.apache.shardingsphere.infra.config.algorithm.AlgorithmConfiguration;
import org.apache.shardingsphere.infra.config.props.ConfigurationProperties;
import org.apache.shardingsphere.infra.database.core.type.DatabaseType;
import org.apache.shardingsphere.infra.hint.HintValueContext;
import org.apache.shardingsphere.infra.instance.ComputeNodeInstanceContext;
import org.apache.shardingsphere.infra.metadata.ShardingSphereMetaData;
import org.apache.shardingsphere.infra.metadata.database.ShardingSphereDatabase;
import org.apache.shardingsphere.infra.metadata.database.resource.ResourceMetaData;
import org.apache.shardingsphere.infra.metadata.database.rule.RuleMetaData;
import org.apache.shardingsphere.infra.metadata.database.schema.model.ShardingSphereColumn;
import org.apache.shardingsphere.infra.metadata.database.schema.model.ShardingSphereSchema;
import org.apache.shardingsphere.infra.metadata.database.schema.model.ShardingSphereTable;
import org.apache.shardingsphere.infra.route.context.RouteContext;
import org.apache.shardingsphere.infra.route.engine.SQLRouteEngine;
import org.apache.shardingsphere.infra.session.connection.ConnectionContext;
import org.apache.shardingsphere.infra.session.query.QueryContext;
import org.apache.shardingsphere.infra.spi.type.typed.TypedSPILoader;
import org.apache.shardingsphere.parser.config.SQLParserRuleConfiguration;
import org.apache.shardingsphere.parser.rule.SQLParserRule;
import org.apache.shardingsphere.sharding.api.config.ShardingRuleConfiguration;
import org.apache.shardingsphere.sharding.api.config.rule.ShardingTableRuleConfiguration;
import org.apache.shardingsphere.sharding.api.config.strategy.sharding.StandardShardingStrategyConfiguration;
import org.apache.shardingsphere.sharding.rule.ShardingRule;
import org.apache.shardingsphere.single.api.config.SingleRuleConfiguration;
import org.apache.shardingsphere.single.rule.SingleRule;
import org.apache.shardingsphere.sql.parser.api.CacheOption;
import org.apache.shardingsphere.test.fixture.jdbc.MockedDataSource;
import org.apache.shardingsphere.test.util.PropertiesBuilder;
import org.apache.shardingsphere.test.util.PropertiesBuilder.Property;
import org.junit.jupiter.api.Test;

import javax.sql.DataSource;
import java.sql.Types;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.mockito.Mockito.RETURNS_DEEP_STUBS;
import static org.mockito.Mockito.mock;

class SubqueryNonBindingTableRouteTest {

private static final String SCHEMA_NAME = "public";

private static final DatabaseType DATABASE_TYPE = TypedSPILoader.getService(DatabaseType.class, "PostgreSQL");

@Test
void assertSubqueryRouteWithNonBindingTables() {
String sql = "SELECT COUNT(t.id) FROM t_order t "
+ "WHERE t.id IN (SELECT o.order_id FROM t_order_item o WHERE o.order_id = ?) "
+ "AND t.order_id = ?";
RouteContext result = executeRoute(sql, Arrays.asList(1L, 1L));
// Expected: 1, Actual (before fix): 8 (2 datasources × 4 table shards)
assertEquals(1, result.getRouteUnits().size());
}

private RouteContext executeRoute(final String sql, final java.util.List params) {
ResourceMetaData resourceMetaData = mock(ResourceMetaData.class, RETURNS_DEEP_STUBS);
ShardingSphereDatabase database = createDatabase(resourceMetaData);
ShardingRule shardingRule = createShardingRule();
SingleRule singleRule = new SingleRule(new SingleRuleConfiguration(), SCHEMA_NAME, DATABASE_TYPE,
createDataSourceMap(), Collections.singleton(shardingRule));
ShardingSphereMetaData metaData = new ShardingSphereMetaData(
Collections.singleton(database), resourceMetaData, mock(RuleMetaData.class), mock(ConfigurationProperties.class));
SQLStatementContext sqlStatementContext = new SQLBindEngine(metaData, SCHEMA_NAME, new HintValueContext())
.bind(new SQLParserRule(new SQLParserRuleConfiguration(new CacheOption(128, 1024L), new CacheOption(128, 1024L)))
.getSQLParserEngine(DATABASE_TYPE).parse(sql, false));
ConnectionContext connectionContext = new ConnectionContext(Collections::emptySet);
connectionContext.setCurrentDatabaseName(SCHEMA_NAME);
QueryContext queryContext = new QueryContext(sqlStatementContext, sql, params, new HintValueContext(), connectionContext, metaData);
return new SQLRouteEngine(Arrays.asList(shardingRule, singleRule), new ConfigurationProperties(new Properties()))
.route(queryContext, mock(RuleMetaData.class), database);
}

private ShardingRule createShardingRule() {
ShardingRuleConfiguration config = new ShardingRuleConfiguration();
ShardingTableRuleConfiguration orderRule = new ShardingTableRuleConfiguration("t_order", "ds_${0..1}.t_order");
orderRule.setDatabaseShardingStrategy(new StandardShardingStrategyConfiguration("order_id", "ds_inline"));
ShardingTableRuleConfiguration orderItemRule = new ShardingTableRuleConfiguration("t_order_item", "ds_${0..1}.t_order_item_${0..3}");
orderItemRule.setDatabaseShardingStrategy(new StandardShardingStrategyConfiguration("order_id", "ds_inline"));
orderItemRule.setTableShardingStrategy(new StandardShardingStrategyConfiguration("order_id", "table_inline"));
config.getTables().add(orderRule);
config.getTables().add(orderItemRule);
config.getShardingAlgorithms().put("ds_inline",
new AlgorithmConfiguration("INLINE", PropertiesBuilder.build(new Property("algorithm-expression", "ds_${order_id % 2}"))));
config.getShardingAlgorithms().put("table_inline",
new AlgorithmConfiguration("INLINE", PropertiesBuilder.build(new Property("algorithm-expression", "t_order_item_${order_id % 4}"))));
return new ShardingRule(config, createDataSourceMap(), mock(ComputeNodeInstanceContext.class), Collections.emptyList());
}

private ShardingSphereDatabase createDatabase(final ResourceMetaData resourceMetaData) {
ShardingSphereTable orderTable = new ShardingSphereTable("t_order",
Arrays.asList(
new ShardingSphereColumn("id", Types.BIGINT, true, false, false, true, false, false),
new ShardingSphereColumn("order_id", Types.BIGINT, false, false, false, false, false, false)),
Collections.emptyList(), Collections.emptyList());
ShardingSphereTable orderItemTable = new ShardingSphereTable("t_order_item",
Arrays.asList(
new ShardingSphereColumn("id", Types.BIGINT, true, false, false, true, false, false),
new ShardingSphereColumn("order_id", Types.BIGINT, false, false, false, false, false, false)),
Collections.emptyList(), Collections.emptyList());
ShardingSphereSchema schema = new ShardingSphereSchema(SCHEMA_NAME, DATABASE_TYPE,
Arrays.asList(orderTable, orderItemTable), Collections.emptyList());
return new ShardingSphereDatabase(SCHEMA_NAME, DATABASE_TYPE, resourceMetaData,
new RuleMetaData(Collections.emptyList()), Collections.singleton(schema));
}

private Map createDataSourceMap() {
Map result = new HashMap<>(2);
result.put("ds_0", new MockedDataSource());
result.put("ds_1", new MockedDataSource());
return result;
}
}
```

Contributor guide

Open the contributing guide

Research direction

Start by reading ShardingStandardRouteEngine.routeByShardingConditionsWithCondition() and routeByMixedConditionsWithCondition(), focusing on how ShardingCondition entries are routed for non-binding tables. Run the supplied SubqueryNonBindingTableRouteTest reproduction and compare its route-unit count with the expected single route. Done means the subquery avoids full-route broadcast while existing routing tests continue to pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, postgresql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.