matrixorigin / matrixorigin/matrixone

Feature Not Supported: Correlated subquery in FROM clause not yet implemented

Open
#23,084 0 comments 0 reactions 1 assignee Assigned to @XuPeng-SH View on GitHub
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## Description

MatrixOne does not support correlated subqueries in the FROM clause (derived tables). When a correlated subquery is used in a SELECT clause and its FROM clause contains a derived table that references the outer query, MatrixOne returns an error indicating that this feature is not yet implemented.

## Error Message

```
ERROR 20102 (HY000) at line 1: correlated subquery in FROM clause is not yet implemented
```

## Affected Cases

This issue affects **2 test cases** from our nested subquery compatibility testing on the `industry_radar_test` database.

## Related Table DDL

### Table: company

```sql
CREATE TABLE `company` (
`id` int NOT NULL,
`full_name` varchar(200) DEFAULT NULL,
`short_name` varchar(128) DEFAULT NULL,
`update_time` datetime NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp() COMMENT '最后修改时间',
`address` varchar(255) DEFAULT NULL,
`board_id` int DEFAULT NULL,
`business_scope` text DEFAULT NULL,
`city` varchar(32) DEFAULT NULL,
`company_profile` text DEFAULT NULL,
`company_type_tags` varchar(255) DEFAULT NULL,
`country` varchar(20) DEFAULT NULL,
`directory_label` varchar(512) DEFAULT NULL,
`district` varchar(32) DEFAULT NULL,
`email` varchar(255) DEFAULT NULL,
`english_name` varchar(255) DEFAULT NULL,
`enterprises_type` varchar(50) DEFAULT NULL,
`founded_time` datetime(6) DEFAULT NULL,
`industry` varchar(32) DEFAULT NULL,
`industry_paqu` varchar(1000) DEFAULT NULL,
`insured_num` varchar(32) DEFAULT NULL,
`product_label` varchar(512) DEFAULT NULL,
`province` varchar(32) DEFAULT NULL,
`registered_capital` varchar(50) DEFAULT NULL,
`representative` varchar(32) DEFAULT NULL,
`taxpayer_num` varchar(50) DEFAULT NULL,
`telephone` varchar(255) DEFAULT NULL,
`website` varchar(500) DEFAULT NULL,
`industry_web_label` varchar(512) DEFAULT NULL COMMENT '国标行业标签',
`industry_domain_label` varchar(512) DEFAULT NULL COMMENT '素问产业链标签',
`scale` varchar(255) DEFAULT NULL COMMENT '规模标签',
`policy_type` varchar(512) DEFAULT NULL COMMENT '政策类型标签',
PRIMARY KEY (`id`),
UNIQUE KEY `company_full_name_idx` (`full_name`),
UNIQUE KEY `company_short_name_idx` (`short_name`),
KEY `company_board_id_idx` (`board_id`),
KEY `company_city_idx` (`city`),
KEY `company_province_idx` (`province`)
)
```

### Table: company_patent

```sql
CREATE TABLE `company_patent` (
`id` int NOT NULL,
`company_id` int DEFAULT NULL,
`name` varchar(512) DEFAULT NULL,
`status` varchar(50) DEFAULT NULL,
`patent_type` varchar(50) DEFAULT NULL,
`num` varchar(100) DEFAULT NULL,
`public_date` date DEFAULT NULL,
`application_date` date DEFAULT NULL,
`update_time` datetime NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp() COMMENT '最后修改时间',
PRIMARY KEY (`id`),
KEY `company_id_idx` (`company_id`),
KEY `status_idx` (`status`)
)
```

## Failing SQL Statements

### Example 1: Correlated subquery with derived table in FROM

```sql
SELECT province,
COUNT(*) as company_count,
(SELECT AVG(patent_count)
FROM (
SELECT company_id, COUNT(*) as patent_count
FROM company_patent
WHERE company_id IN (
SELECT id FROM company WHERE province = c.province
)
GROUP BY company_id
) as province_patents
) as avg_patents_per_company
FROM company c
WHERE province IS NOT NULL
GROUP BY province
LIMIT 10;
```

**Error**: `ERROR 20102 (HY000): correlated subquery in FROM clause is not yet implemented`

### Example 2: Math functions with correlated subquery in FROM

```sql
SELECT c.full_name,
(SELECT ROUND(AVG(patent_count), 2)
FROM (
SELECT company_id, COUNT(*) as patent_count
FROM company_patent
WHERE company_id IN (
SELECT id FROM company WHERE province = c.province
)
GROUP BY company_id
) as province_stats
) as avg_patents
FROM company c
WHERE c.id > (
SELECT FLOOR(AVG(id))
FROM company
)
LIMIT 10;
```

**Error**: `ERROR 20102 (HY000): correlated subquery in FROM clause is not yet implemented`

## Expected Behavior

In MySQL, correlated subqueries can be used in SELECT clauses where the FROM clause contains a derived table that references the outer query. This is a standard feature for complex analytical queries.

## Impact

- **MySQL Compatibility**: Correlated subqueries in FROM clauses are a standard MySQL feature
- **Migration**: Applications using this pattern will fail when migrating to MatrixOne
- **Functionality**: Users cannot perform complex aggregations with correlated subqueries in derived tables
- **Analytical Queries**: Limits the ability to write sophisticated analytical queries

## Test Context

This issue was discovered during nested subquery compatibility testing using the `industry_radar_test` database, which contains:
- `company` table with ~766K records
- `company_patent` table with ~1.2M records

The pattern involves:
1. A correlated subquery in the SELECT clause
2. A derived table in the FROM clause of the subquery
3. The derived table references the outer query (`c.province`)

## Suggested Fix

1. **Parser Support**: Update the SQL parser to recognize and parse correlated subqueries in FROM clauses
2. **Query Planning**: Implement query planning logic to handle correlated subqueries with derived tables
3. **Execution Engine**: Implement execution logic for correlated subqueries in FROM clauses
4. **Correlation Handling**: Ensure proper handling of outer query references within derived tables
5. **Documentation**: Update documentation to reflect support for correlated subqueries in FROM clauses
6. **Testing**: Add comprehensive tests for various patterns of correlated subqueries in FROM clauses

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.