matrixorigin / matrixorigin/matrixone

[Bug]: unflushed CLUSTER BY time-series reads linearly scan CN logtail state

Open
#26,996 1 comment 0 reactions 1 assignee Claimed by @gouhongshen View on GitHub
deferred kind/bug
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## Is there an existing issue for the same bug?

- [x] I have checked the existing issues.

## Branch Name

main

## Commit ID

c3ba9c7df14388956cb38d78681c6c35b75c42b8

## Other Environment Information

- Hardware parameters: Intel Core i7-11700, 8 cores / 16 CPUs, 31 GiB RAM
- OS type: Debian Linux 6.12.43+deb13-amd64
- Others: single CN/TN on `xupeng@10.222.1.55`; MatrixOne standalone; table has no primary key and uses `CLUSTER BY(tenant_id,device_id,metric_id,event_ts)`.

## Actual Behavior

For committed rows that have not been flushed into persisted objects, a selective time-series predicate on a composite `CLUSTER BY` key scans all rows retained by CN logtail replay.

The query-side `PartitionState` is populated from logtail. Its row B-tree is ordered by `BlockID`, `RowID`, and commit timestamp. The in-memory fast path only creates a `rowPrimaryKeyIndex` iterator when the predicate can be converted to a primary-key `MemPKFilter`. This table deliberately has no primary key, so the predicate on the composite cluster key falls back to `NewRowsIter`, which linearly visits all visible logtail rows. The cluster-key ordering is therefore not usable for CN logtail pruning.

Observed before flush, for a query returning two rows from 100,000 ingested rows:

- `EXPLAIN ANALYZE`: `inputBlocks=13`, `inputRows=100000`, `outputRows=2`, `ReadSize=0`
- Repeated polling for 60 seconds did not reduce the scan; it remained 13 blocks / 100,000 rows.
- Median query latency was about 44-45 ms.

`ReadSize=0` confirms the scan is of in-memory/logtail-replayed data rather than persisted object reads.

## Expected Behavior

A selective predicate on a sorted composite `CLUSTER BY` key should avoid linearly scanning all committed rows in CN logtail state. The query must remain strongly consistent and include unflushed committed rows, but the CN replay state should expose a cluster-key-aware seek/pruning structure (or equivalent bounded metadata) for this path.

## Steps to Reproduce

1. Start a standalone MatrixOne service on the commit above.
2. Create a table without a primary key:

```sql
CREATE DATABASE ts_harness;
CREATE TABLE ts_harness.metrics (
tenant_id INT,
device_id INT,
metric_id INT,
event_ts DATETIME,
value DOUBLE,
quality SMALLINT
) CLUSTER BY(tenant_id,device_id,metric_id,event_ts);
```

3. Ingest 100,000 committed, out-of-order time-series rows across 1,000 devices and 5 metrics. Do not execute `mo_ctl('dn','flush', ...)`.
4. Run:

```sql
EXPLAIN ANALYZE
SELECT count(*), sum(value)
FROM ts_harness.metrics
WHERE tenant_id = 0
AND device_id = 0
AND metric_id = 0
AND event_ts >= '2026-01-01 00:03:00';
```

5. Observe that all 100,000 logtail rows are scanned despite only two rows qualifying. Re-run for 60 seconds; pruning does not appear.

Control experiment: execute `SELECT mo_ctl('dn','flush','ts_harness.metrics');` and run the same query. It scans one persisted block (8,192 rows) and returns in roughly 19-27 ms. This control is not a workaround for strong-consistency reads; it demonstrates the missing pruning is specific to the CN logtail path.

## Additional information

Relevant code path:

- `pkg/vm/engine/disttae/logtailreplay/partition_state.go`: logtail inserts populate `PartitionState.rows` and `rowPrimaryKeyIndex` using the primary-key sequence number.
- `pkg/vm/engine/disttae/logtailreplay/types.go`: `PartitionState.rows` is ordered by `BlockID`, `RowID`, and commit timestamp, not the cluster key.
- `pkg/vm/engine/disttae/local_disttae_datasource.go`: non-primary-key reads create `NewRowsIter`, while only a valid `MemPKFilter` gets `NewPrimaryKeyIterWithFilters`.

Existing issue #18603 is a broad reader/range performance request; this report records the concrete, reproducible logtail cluster-key pruning gap.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.