apache / apache/shardingsphere
Decoupling of data sources from sharding rule
- Dominant language
- Java
- Stars
- 20.8k
- Forks
- 6.9k
- Avg merge
- 11h 35m
- Merged PRs (30d)
- 326
Description
## Feature Request
### Is your feature request related to a problem?
When we add a new data source, the user has to update all the sharding rules to include the newly created data source.
**Example:**
```
CREATE SHARDING TABLE RULE t_orders (
DATANODES("ds_${0..1}.t_orders"),
DATABASE_STRATEGY(TYPE="standard", SHARDING_COLUMN=user_id, SHARDING_ALGORITHM(TYPE(NAME="inline",
PROPERTIES("algorithm-expression"="ds_${user_id%2}")))),
KEY_GENERATE_STRATEGY(COLUMN=record_id, KEY_GENERATOR=snowflake_key_generator)
);
CREATE SHARDING TABLE RULE t_invoices (
DATANODES("ds_${0..1}.t_invoices"),
DATABASE_STRATEGY(TYPE="standard", SHARDING_COLUMN=user_id, SHARDING_ALGORITHM(TYPE(NAME="inline",
PROPERTIES("algorithm-expression"="ds_${user_id%2}")))),
KEY_GENERATE_STRATEGY(COLUMN=record_id, KEY_GENERATOR=snowflake_key_generator)
);
```
When we add a new data source, `ds_2`, we need to update the same in both the `t_orders` and `t_invoices` sharding rules. It may look fine for a few tables, but it feels mundane for databases with hundreds of sharding rules.
### Describe the feature you would like.
The problem is mainly because of the tight coupling between the sharding rule and data sources. It's ideal to have some sort of indirection between the sharding rule and data sources.
**Proposal:**
Introduce a layer in the middle that can abstract the data sources.

```
CREATE DATASOURCE GROUP t_orders_group (
DATANODES("ds_${0..1}")
);
```
```
CREATE SHARDING TABLE RULE t_orders (
DATASOURCE_GROUP("t_orders_group.t_orders"),
DATABASE_STRATEGY(TYPE="standard", SHARDING_COLUMN=user_id, SHARDING_ALGORITHM(TYPE(NAME="inline",
PROPERTIES("algorithm-expression"="ds_${user_id%2}")))),
KEY_GENERATE_STRATEGY(COLUMN=record_id, KEY_GENERATOR=snowflake_key_generator)
);
```
Contributor guide
Research direction
The issue names no files, tests, or entry points to begin with. First clarify the intended data-source-group syntax and implementation scope; done would mean sharding rules can reference a group whose data sources can be extended without editing each rule.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100