matrixorigin / matrixorigin/matrixone

MySQL Compatibility: GROUP BY not supported in recursive CTE

Open
#23,077 0 comments 0 reactions 1 assignee Assigned to @XuPeng-SH View on GitHub
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## Description

MatrixOne does not support `GROUP BY` clauses within recursive CTE (Common Table Expression) queries. When using `GROUP BY` in either the anchor part or the recursive part of a recursive CTE, MatrixOne returns a syntax error indicating that GROUP BY is not supported in recursive CTE.

## Error Message

```
ERROR 1064 (HY000) at line 1: SQL parser error: not support group by in recursive cte: 'group by id.level'
```

## Affected Cases

This issue affects **2 test cases** from our recursive CTE compatibility testing on the `industry_radar_test` database.

## Related Table DDL

### Table: industry_domain

```sql
CREATE TABLE `industry_domain` (
`id` int NOT NULL,
`name` varchar(50) DEFAULT NULL,
`desc` varchar(512) DEFAULT NULL,
`father_id` int DEFAULT NULL,
`downstream_id` int DEFAULT NULL,
`is_root` smallint DEFAULT NULL,
`level` int DEFAULT NULL,
`update_time` datetime NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp() COMMENT '最后修改时间',
`guobiao_category` varchar(128) DEFAULT NULL COMMENT '国标行业门类',
`guobiao_major_category` varchar(128) DEFAULT NULL COMMENT '国标行业大类',
`guobiao_medium_category` varchar(128) DEFAULT NULL COMMENT '国标行业中类',
`guobiao_small_category` varchar(128) DEFAULT NULL COMMENT '国标行业小类',
PRIMARY KEY (`id`),
UNIQUE KEY `unique_idx` (`name`,`father_id`,`level`)
)
```

### Table: company

```sql
CREATE TABLE `company` (
`id` int NOT NULL,
`full_name` varchar(200) DEFAULT NULL,
`short_name` varchar(128) DEFAULT NULL,
`update_time` datetime NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp() COMMENT '最后修改时间',
`address` varchar(255) DEFAULT NULL,
`board_id` int DEFAULT NULL,
`business_scope` text DEFAULT NULL,
`city` varchar(32) DEFAULT NULL,
`company_profile` text DEFAULT NULL,
`company_type_tags` varchar(255) DEFAULT NULL,
`country` varchar(20) DEFAULT NULL,
`directory_label` varchar(512) DEFAULT NULL,
`district` varchar(32) DEFAULT NULL,
`email` varchar(255) DEFAULT NULL,
`english_name` varchar(255) DEFAULT NULL,
`enterprises_type` varchar(50) DEFAULT NULL,
`founded_time` datetime(6) DEFAULT NULL,
`industry` varchar(32) DEFAULT NULL,
`industry_paqu` varchar(1000) DEFAULT NULL,
`insured_num` varchar(32) DEFAULT NULL,
`product_label` varchar(512) DEFAULT NULL,
`province` varchar(32) DEFAULT NULL,
`registered_capital` varchar(50) DEFAULT NULL,
`representative` varchar(32) DEFAULT NULL,
`taxpayer_num` varchar(50) DEFAULT NULL,
`telephone` varchar(255) DEFAULT NULL,
`website` varchar(500) DEFAULT NULL,
`industry_web_label` varchar(512) DEFAULT NULL COMMENT '国标行业标签',
`industry_domain_label` varchar(512) DEFAULT NULL COMMENT '素问产业链标签',
`scale` varchar(255) DEFAULT NULL COMMENT '规模标签',
`policy_type` varchar(512) DEFAULT NULL COMMENT '政策类型标签',
PRIMARY KEY (`id`),
UNIQUE KEY `company_full_name_idx` (`full_name`),
UNIQUE KEY `company_short_name_idx` (`short_name`),
KEY `company_board_id_idx` (`board_id`),
KEY `company_city_idx` (`city`),
KEY `company_province_idx` (`province`)
)
```

## Failing SQL Statements

### Example 1: GROUP BY in recursive part

```sql
WITH RECURSIVE level_stats AS (
SELECT level, COUNT(*) as domain_count, 1 as iter
FROM industry_domain
WHERE level = 1
GROUP BY level
UNION ALL
SELECT id.level, COUNT(*) as domain_count, ls.iter + 1
FROM industry_domain id
INNER JOIN level_stats ls ON id.level = ls.level + 1
WHERE ls.iter < 3
GROUP BY id.level
)
SELECT * FROM level_stats ORDER BY level;
```

**Error**: `ERROR 1064 (HY000) at line 1: SQL parser error: not support group by in recursive cte: 'group by id.level'`

### Example 2: GROUP BY in anchor part

```sql
WITH RECURSIVE province_companies AS (
SELECT province, COUNT(*) as cnt, 1 as level
FROM company
WHERE province IS NOT NULL
GROUP BY province
HAVING COUNT(*) > 1000
LIMIT 1
UNION ALL
SELECT c.province, COUNT(*) as cnt, pc.level + 1
FROM company c
INNER JOIN province_companies pc ON c.province = pc.province
WHERE pc.level < 2
GROUP BY c.province
)
SELECT * FROM province_companies LIMIT 10;
```

**Error**: `ERROR 1064 (HY000) at line 1: SQL parser error: syntax error near "UNION ALL"` (GROUP BY in anchor part causes syntax error)

## Expected Behavior

In MySQL, `GROUP BY` is fully supported in both the anchor part and the recursive part of recursive CTEs. This allows for aggregating data at each level of recursion, which is essential for hierarchical data analysis and statistical computations.

## Impact

- **MySQL Compatibility**: GROUP BY in recursive CTEs is a standard MySQL feature
- **Migration**: Applications using GROUP BY in recursive CTEs will fail when migrating to MatrixOne
- **Functionality**: Users cannot perform aggregations within recursive CTEs, limiting analytical capabilities
- **Hierarchical Analysis**: Cannot compute statistics at each level of a hierarchy using recursive CTEs

## Test Context

This issue was discovered during recursive CTE compatibility testing using the `industry_radar_test` database, which contains:
- `industry_domain` table with ~1.5K records (hierarchical structure)
- `company` table with ~766K records

## Suggested Fix

1. **Parser Update**: Update the SQL parser to allow GROUP BY clauses in recursive CTE anchor and recursive parts
2. **Query Planning**: Ensure the query planner can handle GROUP BY within recursive CTE execution
3. **Execution Engine**: Implement proper aggregation handling during recursive CTE iteration
4. **Documentation**: Update documentation to reflect support for GROUP BY in recursive CTEs

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.