matrixorigin / matrixorigin/matrixone
[Bug]: ClickBench Q33 is significantly slower in MatrixOne than DuckDB
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Parent issue
This is a sub-issue of #27665: [significant gap in ClickBench query performance between MatrixOne and DuckDB](https://github.com/matrixorigin/matrixone/issues/27665).
## Summary
ClickBench **Q33** shows a significant MatrixOne performance gap against the DuckDB reference. The values below are elapsed seconds. Each result array is the three measurements emitted by the adapter: `[run 1, run 2, run 3]`; `warm median` is the median of the available run-2/run-3 values.
## Query
```sql
SELECT WatchID, ClientIP, COUNT(*) AS c, SUM(IsRefresh), AVG(ResolutionWidth) FROM hits GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
```
## Performance comparison
| Engine | Run 1 | Run 2 | Run 3 | Warm median |
| --- | ---: | ---: | ---: | ---: |
| MatrixOne | 14.210 | 4.750 | 4.420 | 4.585 |
| DuckDB | 6.187 | 1.248 | 1.277 | 1.262 |
- MatrixOne / DuckDB warm-median ratio: **3.63x**
- Warm-median difference: **3.322 seconds**
- Raw MatrixOne timing array: `[14.210, 4.750, 4.420]`
- Raw DuckDB timing array: `[6.187, 1.248, 1.277]`
## Environment and data
MatrixOne measurement:
- Branch: `main`
- Commit: `249a458ba567b26d1818fb0496b449135cb05ef2`
- Hardware: `server 129`
- OS: CentOS
- Dataset: ClickBench `hits`, **99,997,497 rows**
- MatrixOne raw output: `/mnt/fastdata/ClickBench/matrixone/raw_results.md`
DuckDB reference:
- Result file: `duckdb/results/20260511/c6a.metal.json`
- Machine label: `c6a.metal`
- Result date: `2026-05-11`
The two measurements were not collected on the same hardware. The DuckDB value is the latest local `c6a.metal` reference available; an apples-to-apples rerun on the same server is recommended before assigning a final performance regression magnitude.
## Steps to reproduce
1. Follow the reproduction steps in [#27665](https://github.com/matrixorigin/matrixone/issues/27665), including the MatrixOne and DuckDB adapter READMEs:
- [MatrixOne ClickBench README](https://github.com/matrixorigin/mo-benchmark/blob/main/clickbench/matrixone/README.md#run-in-129)
- [DuckDB ClickBench README](https://github.com/matrixorigin/mo-benchmark/blob/main/clickbench/duckdb/README.md#%E8%BF%90%E8%A1%8C%E5%AE%8C%E6%95%B4-clickbench)
2. Use the same `hits.tsv.gz` dataset for both engines (the adapters support loading the gzip file directly).
3. In the MatrixOne adapter, run `./benchmark.sh` after setting `MATRIXONE_HOME`, `MATRIXONE_DATA_FILE`, `MATRIXONE_HOST`, `MATRIXONE_PORT`, `MATRIXONE_USER`, and `MATRIXONE_PASSWORD` as described in #27665.
4. In the DuckDB adapter, run `./benchmark.sh` after setting `DUCKDB_BIN_PATH`, `DUCKDB_DB_PATH`, and `DUCKDB_DATA_PATH` as described in the README.
5. Compare the Q33 entry in `queries.sql`; run the query three times after loading the data and record elapsed time and whether the query completes successfully.
## Notes
- The timing gap is the subject of this sub-issue; query-result correctness should be verified independently.
## Expected behavior
On comparable hardware with the same dataset, MatrixOne should not exhibit a large query-level performance gap for this workload. Please investigate the execution plan, operators, memory use, and any spill or connection/timeout behavior relevant to this query.
Contributor guide
Assessment
This issue has not been assessed yet.