matrixorigin / matrixorigin/matrixone
Feature Not Supported: Correlated subquery in FROM clause not yet implemented
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Description
MatrixOne does not support correlated subqueries in the FROM clause (derived tables). When a correlated subquery is used in a SELECT clause and its FROM clause contains a derived table that references the outer query, MatrixOne returns an error indicating that this feature is not yet implemented.
## Error Message
```
ERROR 20102 (HY000) at line 1: correlated subquery in FROM clause is not yet implemented
```
## Affected Cases
This issue affects **2 test cases** from our nested subquery compatibility testing on the `industry_radar_test` database.
## Related Table DDL
### Table: company
```sql
CREATE TABLE `company` (
`id` int NOT NULL,
`full_name` varchar(200) DEFAULT NULL,
`short_name` varchar(128) DEFAULT NULL,
`update_time` datetime NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp() COMMENT '最后修改时间',
`address` varchar(255) DEFAULT NULL,
`board_id` int DEFAULT NULL,
`business_scope` text DEFAULT NULL,
`city` varchar(32) DEFAULT NULL,
`company_profile` text DEFAULT NULL,
`company_type_tags` varchar(255) DEFAULT NULL,
`country` varchar(20) DEFAULT NULL,
`directory_label` varchar(512) DEFAULT NULL,
`district` varchar(32) DEFAULT NULL,
`email` varchar(255) DEFAULT NULL,
`english_name` varchar(255) DEFAULT NULL,
`enterprises_type` varchar(50) DEFAULT NULL,
`founded_time` datetime(6) DEFAULT NULL,
`industry` varchar(32) DEFAULT NULL,
`industry_paqu` varchar(1000) DEFAULT NULL,
`insured_num` varchar(32) DEFAULT NULL,
`product_label` varchar(512) DEFAULT NULL,
`province` varchar(32) DEFAULT NULL,
`registered_capital` varchar(50) DEFAULT NULL,
`representative` varchar(32) DEFAULT NULL,
`taxpayer_num` varchar(50) DEFAULT NULL,
`telephone` varchar(255) DEFAULT NULL,
`website` varchar(500) DEFAULT NULL,
`industry_web_label` varchar(512) DEFAULT NULL COMMENT '国标行业标签',
`industry_domain_label` varchar(512) DEFAULT NULL COMMENT '素问产业链标签',
`scale` varchar(255) DEFAULT NULL COMMENT '规模标签',
`policy_type` varchar(512) DEFAULT NULL COMMENT '政策类型标签',
PRIMARY KEY (`id`),
UNIQUE KEY `company_full_name_idx` (`full_name`),
UNIQUE KEY `company_short_name_idx` (`short_name`),
KEY `company_board_id_idx` (`board_id`),
KEY `company_city_idx` (`city`),
KEY `company_province_idx` (`province`)
)
```
### Table: company_patent
```sql
CREATE TABLE `company_patent` (
`id` int NOT NULL,
`company_id` int DEFAULT NULL,
`name` varchar(512) DEFAULT NULL,
`status` varchar(50) DEFAULT NULL,
`patent_type` varchar(50) DEFAULT NULL,
`num` varchar(100) DEFAULT NULL,
`public_date` date DEFAULT NULL,
`application_date` date DEFAULT NULL,
`update_time` datetime NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp() COMMENT '最后修改时间',
PRIMARY KEY (`id`),
KEY `company_id_idx` (`company_id`),
KEY `status_idx` (`status`)
)
```
## Failing SQL Statements
### Example 1: Correlated subquery with derived table in FROM
```sql
SELECT province,
COUNT(*) as company_count,
(SELECT AVG(patent_count)
FROM (
SELECT company_id, COUNT(*) as patent_count
FROM company_patent
WHERE company_id IN (
SELECT id FROM company WHERE province = c.province
)
GROUP BY company_id
) as province_patents
) as avg_patents_per_company
FROM company c
WHERE province IS NOT NULL
GROUP BY province
LIMIT 10;
```
**Error**: `ERROR 20102 (HY000): correlated subquery in FROM clause is not yet implemented`
### Example 2: Math functions with correlated subquery in FROM
```sql
SELECT c.full_name,
(SELECT ROUND(AVG(patent_count), 2)
FROM (
SELECT company_id, COUNT(*) as patent_count
FROM company_patent
WHERE company_id IN (
SELECT id FROM company WHERE province = c.province
)
GROUP BY company_id
) as province_stats
) as avg_patents
FROM company c
WHERE c.id > (
SELECT FLOOR(AVG(id))
FROM company
)
LIMIT 10;
```
**Error**: `ERROR 20102 (HY000): correlated subquery in FROM clause is not yet implemented`
## Expected Behavior
In MySQL, correlated subqueries can be used in SELECT clauses where the FROM clause contains a derived table that references the outer query. This is a standard feature for complex analytical queries.
## Impact
- **MySQL Compatibility**: Correlated subqueries in FROM clauses are a standard MySQL feature
- **Migration**: Applications using this pattern will fail when migrating to MatrixOne
- **Functionality**: Users cannot perform complex aggregations with correlated subqueries in derived tables
- **Analytical Queries**: Limits the ability to write sophisticated analytical queries
## Test Context
This issue was discovered during nested subquery compatibility testing using the `industry_radar_test` database, which contains:
- `company` table with ~766K records
- `company_patent` table with ~1.2M records
The pattern involves:
1. A correlated subquery in the SELECT clause
2. A derived table in the FROM clause of the subquery
3. The derived table references the outer query (`c.province`)
## Suggested Fix
1. **Parser Support**: Update the SQL parser to recognize and parse correlated subqueries in FROM clauses
2. **Query Planning**: Implement query planning logic to handle correlated subqueries with derived tables
3. **Execution Engine**: Implement execution logic for correlated subqueries in FROM clauses
4. **Correlation Handling**: Ensure proper handling of outer query references within derived tables
5. **Documentation**: Update documentation to reflect support for correlated subqueries in FROM clauses
6. **Testing**: Add comprehensive tests for various patterns of correlated subqueries in FROM clauses
Contributor guide
Assessment
This issue has not been assessed yet.