matrixorigin / matrixorigin/matrixone

[Bug]: ClickBench Q35 is significantly slower in MatrixOne than DuckDB

Open
#27,685 5 comments 0 reactions 1 assignee Claimed by @XuPeng-SH View on GitHub
kind/bug needs-triage
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## Parent issue

This is a sub-issue of #27665: [significant gap in ClickBench query performance between MatrixOne and DuckDB](https://github.com/matrixorigin/matrixone/issues/27665).

## Summary

ClickBench **Q35** shows a significant MatrixOne performance gap against the DuckDB reference. The values below are elapsed seconds. Each result array is the three measurements emitted by the adapter: `[run 1, run 2, run 3]`; `warm median` is the median of the available run-2/run-3 values.

## Query

```sql
SELECT 1, URL, COUNT(*) AS c FROM hits GROUP BY 1, URL ORDER BY c DESC LIMIT 10;
```

## Performance comparison

| Engine | Run 1 | Run 2 | Run 3 | Warm median |
| --- | ---: | ---: | ---: | ---: |
| MatrixOne | 55.370 | 54.120 | 57.060 | 55.590 |
| DuckDB | 15.413 | 1.164 | 1.136 | 1.150 |

- MatrixOne / DuckDB warm-median ratio: **48.34x**
- Warm-median difference: **54.440 seconds**
- Raw MatrixOne timing array: `[55.370, 54.120, 57.060]`
- Raw DuckDB timing array: `[15.413, 1.164, 1.136]`

## Environment and data

MatrixOne measurement:

- Branch: `main`
- Commit: `249a458ba567b26d1818fb0496b449135cb05ef2`
- Hardware: `server 129`
- OS: CentOS
- Dataset: ClickBench `hits`, **99,997,497 rows**
- MatrixOne raw output: `/mnt/fastdata/ClickBench/matrixone/raw_results.md`

DuckDB reference:

- Result file: `duckdb/results/20260511/c6a.metal.json`
- Machine label: `c6a.metal`
- Result date: `2026-05-11`

The two measurements were not collected on the same hardware. The DuckDB value is the latest local `c6a.metal` reference available; an apples-to-apples rerun on the same server is recommended before assigning a final performance regression magnitude.

## Steps to reproduce

1. Follow the reproduction steps in [#27665](https://github.com/matrixorigin/matrixone/issues/27665), including the MatrixOne and DuckDB adapter READMEs:
- [MatrixOne ClickBench README](https://github.com/matrixorigin/mo-benchmark/blob/main/clickbench/matrixone/README.md#run-in-129)
- [DuckDB ClickBench README](https://github.com/matrixorigin/mo-benchmark/blob/main/clickbench/duckdb/README.md#%E8%BF%90%E8%A1%8C%E5%AE%8C%E6%95%B4-clickbench)
2. Use the same `hits.tsv.gz` dataset for both engines (the adapters support loading the gzip file directly).
3. In the MatrixOne adapter, run `./benchmark.sh` after setting `MATRIXONE_HOME`, `MATRIXONE_DATA_FILE`, `MATRIXONE_HOST`, `MATRIXONE_PORT`, `MATRIXONE_USER`, and `MATRIXONE_PASSWORD` as described in #27665.
4. In the DuckDB adapter, run `./benchmark.sh` after setting `DUCKDB_BIN_PATH`, `DUCKDB_DB_PATH`, and `DUCKDB_DATA_PATH` as described in the README.
5. Compare the Q35 entry in `queries.sql`; run the query three times after loading the data and record elapsed time and whether the query completes successfully.

## Notes

- The timing gap is the subject of this sub-issue; query-result correctness should be verified independently.

## Expected behavior

On comparable hardware with the same dataset, MatrixOne should not exhibit a large query-level performance gap for this workload. Please investigate the execution plan, operators, memory use, and any spill or connection/timeout behavior relevant to this query.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.